Note: This is test shard 8 of 8.
[==========] Running 7 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 1 test from TableRangeRebalancingTest
[ RUN ] TableRangeRebalancingTest.Basic
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240501 19:55:58.224277 21320 test_util.cc:268] Using random seed: -536011904
I20240501 19:55:58.230779 21320 ts_itest-base.cc:112] Starting cluster with:
I20240501 19:55:58.230894 21320 ts_itest-base.cc:113] --------------
I20240501 19:55:58.230927 21320 ts_itest-base.cc:114] 3 tablet servers
I20240501 19:55:58.230952 21320 ts_itest-base.cc:115] 3 replicas per TS
I20240501 19:55:58.230978 21320 ts_itest-base.cc:116] --------------
2024-05-01T19:55:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T19:55:58Z Disabled control of system clock
I20240501 19:55:58.240183 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:46537
--webserver_interface=127.20.210.62
--webserver_port=0
--builtin_ntp_servers=127.20.210.20:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.210.62:46537
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240501 19:55:58.340899 21332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:55:58.341111 21332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:55:58.341149 21332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:55:58.344095 21332 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:55:58.344151 21332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 19:55:58.344175 21332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:55:58.344195 21332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 19:55:58.344225 21332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 19:55:58.347625 21332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.20.210.62:46537
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:46537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.20.210.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.21332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:55:58.348383 21332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:55:58.349385 21332 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:55:58.356601 21337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:55:58.357213 21338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:55:58.357928 21340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:55:58.358646 21332 server_base.cc:1034] running on GCE node
I20240501 19:55:58.359069 21332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:55:58.359871 21332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:55:58.361027 21332 hybrid_clock.cc:648] HybridClock initialized: now 1714593358361003 us; error 34 us; skew 500 ppm
I20240501 19:55:58.367002 21332 webserver.cc:460] Webserver started at http://127.20.210.62:44007/ using document root <none> and password file <none>
I20240501 19:55:58.367496 21332 fs_manager.cc:359] Metadata directory not provided
I20240501 19:55:58.367551 21332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:55:58.367705 21332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:55:58.369155 21332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "cb18cab4cd9b458191e1c9bfb8954689"
format_stamp: "Formatted at 2024-05-01 19:55:58 on dist-test-slave-vgh4"
I20240501 19:55:58.369592 21332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "cb18cab4cd9b458191e1c9bfb8954689"
format_stamp: "Formatted at 2024-05-01 19:55:58 on dist-test-slave-vgh4"
I20240501 19:55:58.372663 21332 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20240501 19:55:58.374639 21346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:55:58.375519 21332 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:55:58.375617 21332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
uuid: "cb18cab4cd9b458191e1c9bfb8954689"
format_stamp: "Formatted at 2024-05-01 19:55:58 on dist-test-slave-vgh4"
I20240501 19:55:58.375692 21332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:55:58.404274 21332 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:55:58.404855 21332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:55:58.404997 21332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:55:58.411000 21332 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.62:46537
I20240501 19:55:58.411022 21398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.62:46537 every 8 connection(s)
I20240501 19:55:58.411862 21332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 19:55:58.414022 21399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:55:58.415302 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 21332
I20240501 19:55:58.415434 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 19:55:58.418627 21399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689: Bootstrap starting.
I20240501 19:55:58.421051 21399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689: Neither blocks nor log segments found. Creating new log.
I20240501 19:55:58.421813 21399 log.cc:824] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689: Log is configured to *not* fsync() on all Append() calls
I20240501 19:55:58.423436 21399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689: No bootstrap required, opened a new log
I20240501 19:55:58.426301 21399 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb18cab4cd9b458191e1c9bfb8954689" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 46537 } }
I20240501 19:55:58.426461 21399 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:55:58.426517 21399 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb18cab4cd9b458191e1c9bfb8954689, State: Initialized, Role: FOLLOWER
I20240501 19:55:58.426890 21399 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb18cab4cd9b458191e1c9bfb8954689" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 46537 } }
I20240501 19:55:58.427002 21399 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:55:58.427059 21399 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:55:58.427158 21399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:55:58.427773 21399 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb18cab4cd9b458191e1c9bfb8954689" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 46537 } }
I20240501 19:55:58.428170 21399 leader_election.cc:302] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb18cab4cd9b458191e1c9bfb8954689; no voters:
I20240501 19:55:58.428442 21399 leader_election.cc:288] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:55:58.428788 21404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:55:58.428951 21404 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [term 1 LEADER]: Becoming Leader. State: Replica: cb18cab4cd9b458191e1c9bfb8954689, State: Running, Role: LEADER
I20240501 19:55:58.429198 21404 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb18cab4cd9b458191e1c9bfb8954689" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 46537 } }
I20240501 19:55:58.429482 21399 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 19:55:58.430320 21404 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb18cab4cd9b458191e1c9bfb8954689. Latest consensus state: current_term: 1 leader_uuid: "cb18cab4cd9b458191e1c9bfb8954689" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb18cab4cd9b458191e1c9bfb8954689" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 46537 } } }
I20240501 19:55:58.430593 21404 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [sys.catalog]: This master's current role is: LEADER
I20240501 19:55:58.430963 21413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 19:55:58.430305 21405 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb18cab4cd9b458191e1c9bfb8954689" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb18cab4cd9b458191e1c9bfb8954689" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 46537 } } }
I20240501 19:55:58.431493 21405 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689 [sys.catalog]: This master's current role is: LEADER
I20240501 19:55:58.433652 21413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 19:55:58.438141 21413 catalog_manager.cc:1347] Generated new cluster ID: 6d09673d79724901b025c7306e9eec9d
I20240501 19:55:58.438191 21413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 19:55:58.445441 21413 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 19:55:58.446099 21413 catalog_manager.cc:1504] Loading token signing keys...
I20240501 19:55:58.451022 21413 catalog_manager.cc:5857] T 00000000000000000000000000000000 P cb18cab4cd9b458191e1c9bfb8954689: Generated new TSK 0
I20240501 19:55:58.451535 21413 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 19:55:58.456708 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.1:0
--local_ip_for_outbound_sockets=127.20.210.1
--webserver_interface=127.20.210.1
--webserver_port=0
--tserver_master_addrs=127.20.210.62:46537
--builtin_ntp_servers=127.20.210.20:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 19:55:58.555790 21423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:55:58.556041 21423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:55:58.556111 21423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:55:58.559238 21423 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:55:58.559304 21423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:55:58.559386 21423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.1
I20240501 19:55:58.562736 21423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.20.210.1
--webserver_port=0
--tserver_master_addrs=127.20.210.62:46537
--never_fsync=true
--heap_profile_path=/tmp/kudu.21423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.1
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:55:58.563437 21423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:55:58.564448 21423 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:55:58.572464 21429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:55:58.573002 21428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:55:58.573673 21431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:55:58.574349 21423 server_base.cc:1034] running on GCE node
I20240501 19:55:58.574674 21423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:55:58.575141 21423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:55:58.576282 21423 hybrid_clock.cc:648] HybridClock initialized: now 1714593358576268 us; error 32 us; skew 500 ppm
I20240501 19:55:58.577814 21423 webserver.cc:460] Webserver started at http://127.20.210.1:34003/ using document root <none> and password file <none>
I20240501 19:55:58.578368 21423 fs_manager.cc:359] Metadata directory not provided
I20240501 19:55:58.578431 21423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:55:58.578598 21423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:55:58.580165 21423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "5b2d64c1fbe5462baa64e103f2545ba0"
format_stamp: "Formatted at 2024-05-01 19:55:58 on dist-test-slave-vgh4"
I20240501 19:55:58.580575 21423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "5b2d64c1fbe5462baa64e103f2545ba0"
format_stamp: "Formatted at 2024-05-01 19:55:58 on dist-test-slave-vgh4"
I20240501 19:55:58.583633 21423 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 19:55:58.585569 21437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:55:58.586453 21423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 19:55:58.586551 21423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "5b2d64c1fbe5462baa64e103f2545ba0"
format_stamp: "Formatted at 2024-05-01 19:55:58 on dist-test-slave-vgh4"
I20240501 19:55:58.586624 21423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:55:58.596514 21423 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:55:58.597018 21423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:55:58.597152 21423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:55:58.597584 21423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:55:58.598373 21423 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:55:58.598421 21423 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:55:58.598471 21423 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:55:58.598501 21423 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:55:58.606323 21423 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.1:33105
I20240501 19:55:58.606344 21550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.1:33105 every 8 connection(s)
I20240501 19:55:58.607105 21423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 19:55:58.611524 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 21423
I20240501 19:55:58.611619 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 19:55:58.613639 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.2:0
--local_ip_for_outbound_sockets=127.20.210.2
--webserver_interface=127.20.210.2
--webserver_port=0
--tserver_master_addrs=127.20.210.62:46537
--builtin_ntp_servers=127.20.210.20:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:55:58.615574 21551 heartbeater.cc:347] Connected to a master server at 127.20.210.62:46537
I20240501 19:55:58.615773 21551 heartbeater.cc:464] Registering TS with master...
I20240501 19:55:58.616233 21551 heartbeater.cc:511] Master 127.20.210.62:46537 requested a full tablet report, sending...
I20240501 19:55:58.617628 21363 ts_manager.cc:194] Registered new tserver with Master: 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105)
I20240501 19:55:58.619338 21363 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.1:38863
W20240501 19:55:58.714911 21554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:55:58.715091 21554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:55:58.715129 21554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:55:58.717913 21554 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:55:58.717962 21554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:55:58.718021 21554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.2
I20240501 19:55:58.721308 21554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.20.210.2
--webserver_port=0
--tserver_master_addrs=127.20.210.62:46537
--never_fsync=true
--heap_profile_path=/tmp/kudu.21554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.2
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:55:58.722020 21554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:55:58.723014 21554 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:55:58.730966 21559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:55:58.731611 21560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:55:58.732262 21562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:55:58.732993 21554 server_base.cc:1034] running on GCE node
I20240501 19:55:58.733305 21554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:55:58.733786 21554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:55:58.734925 21554 hybrid_clock.cc:648] HybridClock initialized: now 1714593358734902 us; error 36 us; skew 500 ppm
I20240501 19:55:58.736469 21554 webserver.cc:460] Webserver started at http://127.20.210.2:37599/ using document root <none> and password file <none>
I20240501 19:55:58.736975 21554 fs_manager.cc:359] Metadata directory not provided
I20240501 19:55:58.737031 21554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:55:58.737197 21554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:55:58.738735 21554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
format_stamp: "Formatted at 2024-05-01 19:55:58 on dist-test-slave-vgh4"
I20240501 19:55:58.739185 21554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
format_stamp: "Formatted at 2024-05-01 19:55:58 on dist-test-slave-vgh4"
I20240501 19:55:58.742175 21554 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 19:55:58.744215 21568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:55:58.745111 21554 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240501 19:55:58.745209 21554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
format_stamp: "Formatted at 2024-05-01 19:55:58 on dist-test-slave-vgh4"
I20240501 19:55:58.745285 21554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:55:58.756367 21554 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:55:58.756881 21554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:55:58.757035 21554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:55:58.757493 21554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:55:58.758291 21554 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:55:58.758337 21554 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:55:58.758385 21554 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:55:58.758421 21554 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:55:58.766865 21554 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.2:35565
I20240501 19:55:58.766908 21681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.2:35565 every 8 connection(s)
I20240501 19:55:58.767688 21554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 19:55:58.770437 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 21554
I20240501 19:55:58.770526 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 19:55:58.772526 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.3:0
--local_ip_for_outbound_sockets=127.20.210.3
--webserver_interface=127.20.210.3
--webserver_port=0
--tserver_master_addrs=127.20.210.62:46537
--builtin_ntp_servers=127.20.210.20:46669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:55:58.776037 21682 heartbeater.cc:347] Connected to a master server at 127.20.210.62:46537
I20240501 19:55:58.776232 21682 heartbeater.cc:464] Registering TS with master...
I20240501 19:55:58.776692 21682 heartbeater.cc:511] Master 127.20.210.62:46537 requested a full tablet report, sending...
I20240501 19:55:58.777657 21363 ts_manager.cc:194] Registered new tserver with Master: 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565)
I20240501 19:55:58.778481 21363 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.2:40001
W20240501 19:55:58.872565 21685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:55:58.872727 21685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:55:58.872762 21685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:55:58.875561 21685 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:55:58.875617 21685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:55:58.875675 21685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.3
I20240501 19:55:58.878885 21685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:46669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.20.210.3
--webserver_port=0
--tserver_master_addrs=127.20.210.62:46537
--never_fsync=true
--heap_profile_path=/tmp/kudu.21685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.3
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:55:58.879541 21685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:55:58.880456 21685 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:55:58.888164 21691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:55:58.888895 21693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:55:58.889571 21690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:55:58.890393 21685 server_base.cc:1034] running on GCE node
I20240501 19:55:58.890766 21685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:55:58.891206 21685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:55:58.892350 21685 hybrid_clock.cc:648] HybridClock initialized: now 1714593358892342 us; error 23 us; skew 500 ppm
I20240501 19:55:58.893899 21685 webserver.cc:460] Webserver started at http://127.20.210.3:44811/ using document root <none> and password file <none>
I20240501 19:55:58.894377 21685 fs_manager.cc:359] Metadata directory not provided
I20240501 19:55:58.894429 21685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:55:58.894567 21685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:55:58.896389 21685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "aae2aec0505e468193ac56ee42f6211d"
format_stamp: "Formatted at 2024-05-01 19:55:58 on dist-test-slave-vgh4"
I20240501 19:55:58.896768 21685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "aae2aec0505e468193ac56ee42f6211d"
format_stamp: "Formatted at 2024-05-01 19:55:58 on dist-test-slave-vgh4"
I20240501 19:55:58.899885 21685 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240501 19:55:58.901752 21699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:55:58.902665 21685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:55:58.902752 21685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "aae2aec0505e468193ac56ee42f6211d"
format_stamp: "Formatted at 2024-05-01 19:55:58 on dist-test-slave-vgh4"
I20240501 19:55:58.902835 21685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:55:58.934108 21685 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:55:58.934651 21685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:55:58.934793 21685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:55:58.935240 21685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:55:58.936010 21685 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:55:58.936054 21685 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:55:58.936097 21685 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:55:58.936127 21685 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:55:58.944018 21685 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.3:37115
I20240501 19:55:58.944038 21812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.3:37115 every 8 connection(s)
I20240501 19:55:58.944859 21685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 19:55:58.947969 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 21685
I20240501 19:55:58.948045 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 19:55:58.952489 21813 heartbeater.cc:347] Connected to a master server at 127.20.210.62:46537
I20240501 19:55:58.952654 21813 heartbeater.cc:464] Registering TS with master...
I20240501 19:55:58.953059 21813 heartbeater.cc:511] Master 127.20.210.62:46537 requested a full tablet report, sending...
I20240501 19:55:58.953830 21363 ts_manager.cc:194] Registered new tserver with Master: aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:55:58.954347 21363 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.3:60683
I20240501 19:55:58.961611 21320 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20240501 19:55:58.970990 21320 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20240501 19:55:59.621418 21551 heartbeater.cc:503] Master 127.20.210.62:46537 was elected leader, sending a full tablet report...
I20240501 19:55:59.780452 21682 heartbeater.cc:503] Master 127.20.210.62:46537 was elected leader, sending a full tablet report...
I20240501 19:55:59.955850 21813 heartbeater.cc:503] Master 127.20.210.62:46537 was elected leader, sending a full tablet report...
I20240501 19:56:02.723491 21363 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52214:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\200\007\001\000\000\000\000\000\006\001\000\000\000\000\000\007\001\000\377\377\377\177"
indirect_data: "<redacted>"""
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 8
seed: 0
}
}
W20240501 19:56:02.726364 21363 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240501 19:56:02.745316 21747 tablet_service.cc:1467] Processing CreateTablet for tablet b1061bac480546159dce2050285db8b6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.745437 21609 tablet_service.cc:1467] Processing CreateTablet for tablet f947ac041bd24301ba41e195afe282a7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.746160 21739 tablet_service.cc:1467] Processing CreateTablet for tablet bc0f6426eae845ec8c7ea96580449d59 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.746165 21614 tablet_service.cc:1467] Processing CreateTablet for tablet 1832a822dbe8451ea298baaa319b94e5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.746551 21747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1061bac480546159dce2050285db8b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.746553 21609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f947ac041bd24301ba41e195afe282a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.746593 21740 tablet_service.cc:1467] Processing CreateTablet for tablet f947ac041bd24301ba41e195afe282a7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.746624 21613 tablet_service.cc:1467] Processing CreateTablet for tablet db648e6633204661b39a21199c8786da (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.746850 21613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db648e6633204661b39a21199c8786da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.747020 21611 tablet_service.cc:1467] Processing CreateTablet for tablet f46458ee560a4bd8a1dfc050fe0a76f1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.745553 21737 tablet_service.cc:1467] Processing CreateTablet for tablet 1637a1003436499e9baa39df67ba7bca (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.747257 21611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f46458ee560a4bd8a1dfc050fe0a76f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.747421 21737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1637a1003436499e9baa39df67ba7bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.747702 21612 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9a7537441d4deeae6178e0d06dc532 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.747713 21741 tablet_service.cc:1467] Processing CreateTablet for tablet 50ae742565b241da8372a1791ee7f1b4 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.747941 21741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50ae742565b241da8372a1791ee7f1b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.747963 21612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9a7537441d4deeae6178e0d06dc532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.748457 21610 tablet_service.cc:1467] Processing CreateTablet for tablet 50ae742565b241da8372a1791ee7f1b4 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.748559 21742 tablet_service.cc:1467] Processing CreateTablet for tablet f46458ee560a4bd8a1dfc050fe0a76f1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.748710 21610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50ae742565b241da8372a1791ee7f1b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.748757 21742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f46458ee560a4bd8a1dfc050fe0a76f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.749301 21743 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9a7537441d4deeae6178e0d06dc532 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.749541 21743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9a7537441d4deeae6178e0d06dc532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.745831 21738 tablet_service.cc:1467] Processing CreateTablet for tablet a35bd26ec43741e08fc811450aef8ac6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.750592 21738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35bd26ec43741e08fc811450aef8ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.750654 21744 tablet_service.cc:1467] Processing CreateTablet for tablet db648e6633204661b39a21199c8786da (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.750873 21744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db648e6633204661b39a21199c8786da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.750929 21607 tablet_service.cc:1467] Processing CreateTablet for tablet a35bd26ec43741e08fc811450aef8ac6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.751173 21607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35bd26ec43741e08fc811450aef8ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.751561 21745 tablet_service.cc:1467] Processing CreateTablet for tablet 1832a822dbe8451ea298baaa319b94e5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.751808 21745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1832a822dbe8451ea298baaa319b94e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.745327 21736 tablet_service.cc:1467] Processing CreateTablet for tablet c829a89ebd3242ef9cacc7762ed72912 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.752938 21746 tablet_service.cc:1467] Processing CreateTablet for tablet cc489e219ea8431eafa345e8a6417a9a (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.753034 21606 tablet_service.cc:1467] Processing CreateTablet for tablet 1637a1003436499e9baa39df67ba7bca (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.753108 21736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c829a89ebd3242ef9cacc7762ed72912. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.753254 21606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1637a1003436499e9baa39df67ba7bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.753578 21831 tablet_bootstrap.cc:492] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.754336 21735 tablet_service.cc:1467] Processing CreateTablet for tablet be54f71e60754ba88ab21bd3b92d6947 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.754499 21740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f947ac041bd24301ba41e195afe282a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.755333 21739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc0f6426eae845ec8c7ea96580449d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.755784 21735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be54f71e60754ba88ab21bd3b92d6947. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.755328 21482 tablet_service.cc:1467] Processing CreateTablet for tablet db648e6633204661b39a21199c8786da (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.756115 21831 tablet_bootstrap.cc:654] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.756213 21482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db648e6633204661b39a21199c8786da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.756760 21746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc489e219ea8431eafa345e8a6417a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.756798 21831 log.cc:824] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:02.757838 21484 tablet_service.cc:1467] Processing CreateTablet for tablet cc489e219ea8431eafa345e8a6417a9a (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.758085 21484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc489e219ea8431eafa345e8a6417a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.758270 21831 tablet_bootstrap.cc:492] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.758416 21831 ts_tablet_manager.cc:1367] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240501 19:56:02.758453 21481 tablet_service.cc:1467] Processing CreateTablet for tablet 3c9a7537441d4deeae6178e0d06dc532 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.758674 21481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c9a7537441d4deeae6178e0d06dc532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.759800 21480 tablet_service.cc:1467] Processing CreateTablet for tablet f46458ee560a4bd8a1dfc050fe0a76f1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.760004 21480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f46458ee560a4bd8a1dfc050fe0a76f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.761874 21831 raft_consensus.cc:348] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.762002 21479 tablet_service.cc:1467] Processing CreateTablet for tablet 50ae742565b241da8372a1791ee7f1b4 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.762060 21831 raft_consensus.cc:374] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.762120 21831 raft_consensus.cc:729] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.762275 21479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50ae742565b241da8372a1791ee7f1b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.762472 21833 tablet_bootstrap.cc:492] T cc489e219ea8431eafa345e8a6417a9a P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.763069 21831 consensus_queue.cc:260] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.764333 21833 tablet_bootstrap.cc:654] T cc489e219ea8431eafa345e8a6417a9a P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.764411 21831 ts_tablet_manager.cc:1398] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 19:56:02.764554 21478 tablet_service.cc:1467] Processing CreateTablet for tablet f947ac041bd24301ba41e195afe282a7 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.764767 21478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f947ac041bd24301ba41e195afe282a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.765012 21833 log.cc:824] T cc489e219ea8431eafa345e8a6417a9a P 5b2d64c1fbe5462baa64e103f2545ba0: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:02.765677 21831 tablet_bootstrap.cc:492] T a35bd26ec43741e08fc811450aef8ac6 P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.766469 21831 tablet_bootstrap.cc:654] T a35bd26ec43741e08fc811450aef8ac6 P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.766539 21477 tablet_service.cc:1467] Processing CreateTablet for tablet bc0f6426eae845ec8c7ea96580449d59 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.766760 21477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc0f6426eae845ec8c7ea96580449d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.767699 21831 tablet_bootstrap.cc:492] T a35bd26ec43741e08fc811450aef8ac6 P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.767762 21831 ts_tablet_manager.cc:1367] T a35bd26ec43741e08fc811450aef8ac6 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:02.768203 21831 raft_consensus.cc:348] T a35bd26ec43741e08fc811450aef8ac6 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.768280 21831 raft_consensus.cc:374] T a35bd26ec43741e08fc811450aef8ac6 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.768308 21831 raft_consensus.cc:729] T a35bd26ec43741e08fc811450aef8ac6 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.768433 21831 consensus_queue.cc:260] T a35bd26ec43741e08fc811450aef8ac6 P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.768576 21476 tablet_service.cc:1467] Processing CreateTablet for tablet a35bd26ec43741e08fc811450aef8ac6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.768709 21831 ts_tablet_manager.cc:1398] T a35bd26ec43741e08fc811450aef8ac6 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.768772 21476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35bd26ec43741e08fc811450aef8ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.768850 21831 tablet_bootstrap.cc:492] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.769593 21831 tablet_bootstrap.cc:654] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.771092 21473 tablet_service.cc:1467] Processing CreateTablet for tablet be54f71e60754ba88ab21bd3b92d6947 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.771324 21473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be54f71e60754ba88ab21bd3b92d6947. 1 dirs total, 0 dirs full, 0 dirs failed
W20240501 19:56:02.771395 21683 tablet.cc:2369] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:56:02.771741 21475 tablet_service.cc:1467] Processing CreateTablet for tablet 1637a1003436499e9baa39df67ba7bca (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.772006 21475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1637a1003436499e9baa39df67ba7bca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.771947 21474 tablet_service.cc:1467] Processing CreateTablet for tablet c829a89ebd3242ef9cacc7762ed72912 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.772141 21474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c829a89ebd3242ef9cacc7762ed72912. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.771739 21833 tablet_bootstrap.cc:492] T cc489e219ea8431eafa345e8a6417a9a P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.771411 21831 tablet_bootstrap.cc:492] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.773068 21831 ts_tablet_manager.cc:1367] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240501 19:56:02.773357 21831 raft_consensus.cc:348] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.773423 21831 raft_consensus.cc:374] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.773447 21831 raft_consensus.cc:729] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.773546 21831 consensus_queue.cc:260] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.773716 21831 ts_tablet_manager.cc:1398] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.773916 21833 ts_tablet_manager.cc:1367] T cc489e219ea8431eafa345e8a6417a9a P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20240501 19:56:02.773816 21831 tablet_bootstrap.cc:492] T 1637a1003436499e9baa39df67ba7bca P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.774039 21608 tablet_service.cc:1467] Processing CreateTablet for tablet bc0f6426eae845ec8c7ea96580449d59 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.774281 21608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc0f6426eae845ec8c7ea96580449d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.774662 21831 tablet_bootstrap.cc:654] T 1637a1003436499e9baa39df67ba7bca P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.773963 21605 tablet_service.cc:1467] Processing CreateTablet for tablet c829a89ebd3242ef9cacc7762ed72912 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.775300 21605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c829a89ebd3242ef9cacc7762ed72912. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.776293 21831 tablet_bootstrap.cc:492] T 1637a1003436499e9baa39df67ba7bca P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.776350 21831 ts_tablet_manager.cc:1367] T 1637a1003436499e9baa39df67ba7bca P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:02.776645 21833 raft_consensus.cc:348] T cc489e219ea8431eafa345e8a6417a9a P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.776791 21833 raft_consensus.cc:374] T cc489e219ea8431eafa345e8a6417a9a P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.776759 21831 raft_consensus.cc:348] T 1637a1003436499e9baa39df67ba7bca P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.776839 21831 raft_consensus.cc:374] T 1637a1003436499e9baa39df67ba7bca P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.776871 21831 raft_consensus.cc:729] T 1637a1003436499e9baa39df67ba7bca P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.755530 21483 tablet_service.cc:1467] Processing CreateTablet for tablet 1832a822dbe8451ea298baaa319b94e5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.776964 21831 consensus_queue.cc:260] T 1637a1003436499e9baa39df67ba7bca P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.777130 21483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1832a822dbe8451ea298baaa319b94e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.777204 21831 ts_tablet_manager.cc:1398] T 1637a1003436499e9baa39df67ba7bca P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.776849 21833 raft_consensus.cc:729] T cc489e219ea8431eafa345e8a6417a9a P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.777312 21831 tablet_bootstrap.cc:492] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.745875 21615 tablet_service.cc:1467] Processing CreateTablet for tablet cc489e219ea8431eafa345e8a6417a9a (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.777833 21615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc489e219ea8431eafa345e8a6417a9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.777825 21833 consensus_queue.cc:260] T cc489e219ea8431eafa345e8a6417a9a P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.778097 21831 tablet_bootstrap.cc:654] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.778821 21833 ts_tablet_manager.cc:1398] T cc489e219ea8431eafa345e8a6417a9a P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240501 19:56:02.745317 21604 tablet_service.cc:1467] Processing CreateTablet for tablet be54f71e60754ba88ab21bd3b92d6947 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.779131 21833 tablet_bootstrap.cc:492] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.779271 21614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1832a822dbe8451ea298baaa319b94e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.780009 21833 tablet_bootstrap.cc:654] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.780251 21604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be54f71e60754ba88ab21bd3b92d6947. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.781198 21833 tablet_bootstrap.cc:492] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.781289 21833 ts_tablet_manager.cc:1367] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:02.781790 21833 raft_consensus.cc:348] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.781910 21833 raft_consensus.cc:374] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.781946 21833 raft_consensus.cc:729] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.782056 21833 consensus_queue.cc:260] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.782321 21833 ts_tablet_manager.cc:1398] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.782471 21833 tablet_bootstrap.cc:492] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.755326 21485 tablet_service.cc:1467] Processing CreateTablet for tablet b1061bac480546159dce2050285db8b6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.782626 21485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1061bac480546159dce2050285db8b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.745317 21616 tablet_service.cc:1467] Processing CreateTablet for tablet b1061bac480546159dce2050285db8b6 (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.783195 21616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1061bac480546159dce2050285db8b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.783313 21833 tablet_bootstrap.cc:654] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.784672 21833 tablet_bootstrap.cc:492] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.784808 21833 ts_tablet_manager.cc:1367] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:02.785151 21831 tablet_bootstrap.cc:492] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.785351 21831 ts_tablet_manager.cc:1367] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240501 19:56:02.785581 21833 raft_consensus.cc:348] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.785760 21833 raft_consensus.cc:374] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.785830 21833 raft_consensus.cc:729] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.786011 21833 consensus_queue.cc:260] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.786458 21833 ts_tablet_manager.cc:1398] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:02.786409 21831 raft_consensus.cc:348] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.786692 21833 tablet_bootstrap.cc:492] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.786688 21831 raft_consensus.cc:374] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.787389 21834 tablet_bootstrap.cc:492] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.787611 21831 raft_consensus.cc:729] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.788105 21831 consensus_queue.cc:260] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.788877 21615 tablet_service.cc:1467] Processing CreateTablet for tablet 4516205f081c4ea59bd657d28f24ff9b (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.788930 21485 tablet_service.cc:1467] Processing CreateTablet for tablet 4516205f081c4ea59bd657d28f24ff9b (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.789134 21615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4516205f081c4ea59bd657d28f24ff9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.789139 21485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4516205f081c4ea59bd657d28f24ff9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.790761 21833 tablet_bootstrap.cc:654] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.790992 21604 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa9614626304947b240419d14d82fdb (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.791229 21604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa9614626304947b240419d14d82fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.791877 21736 tablet_service.cc:1467] Processing CreateTablet for tablet 4516205f081c4ea59bd657d28f24ff9b (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240501 19:56:02.791849 21747 tablet_service.cc:1467] Processing CreateTablet for tablet e8ccd935e153446bae5651f2a7165a1b (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.791978 21738 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa9614626304947b240419d14d82fdb (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.792121 21736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4516205f081c4ea59bd657d28f24ff9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.792213 21747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8ccd935e153446bae5651f2a7165a1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.792675 21738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa9614626304947b240419d14d82fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.794449 21616 tablet_service.cc:1467] Processing CreateTablet for tablet e8ccd935e153446bae5651f2a7165a1b (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.794670 21616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8ccd935e153446bae5651f2a7165a1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.795233 21482 tablet_service.cc:1467] Processing CreateTablet for tablet e8ccd935e153446bae5651f2a7165a1b (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.795469 21482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8ccd935e153446bae5651f2a7165a1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.796486 21831 ts_tablet_manager.cc:1398] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.011s user 0.000s sys 0.001s
I20240501 19:56:02.796662 21831 tablet_bootstrap.cc:492] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.797499 21831 tablet_bootstrap.cc:654] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.797633 21485 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa9614626304947b240419d14d82fdb (DEFAULT_TABLE table=rebalance_test_table_0 [id=0c126b33b3c846b0851183783ad6f1ee]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240501 19:56:02.797873 21485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aa9614626304947b240419d14d82fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:02.798671 21831 tablet_bootstrap.cc:492] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.798763 21831 ts_tablet_manager.cc:1367] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 19:56:02.799347 21831 raft_consensus.cc:348] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.799465 21831 raft_consensus.cc:374] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.799499 21831 raft_consensus.cc:729] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.799645 21834 tablet_bootstrap.cc:654] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.799623 21831 consensus_queue.cc:260] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.800112 21833 tablet_bootstrap.cc:492] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.800189 21833 ts_tablet_manager.cc:1367] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240501 19:56:02.800242 21831 ts_tablet_manager.cc:1398] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.800397 21831 tablet_bootstrap.cc:492] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.800588 21833 raft_consensus.cc:348] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.800628 21834 log.cc:824] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:02.800673 21833 raft_consensus.cc:374] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.800704 21833 raft_consensus.cc:729] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.800822 21833 consensus_queue.cc:260] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.801075 21833 ts_tablet_manager.cc:1398] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.801225 21833 tablet_bootstrap.cc:492] T bc0f6426eae845ec8c7ea96580449d59 P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.801353 21831 tablet_bootstrap.cc:654] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.802116 21833 tablet_bootstrap.cc:654] T bc0f6426eae845ec8c7ea96580449d59 P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.802372 21831 tablet_bootstrap.cc:492] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.802460 21831 ts_tablet_manager.cc:1367] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 19:56:02.802925 21831 raft_consensus.cc:348] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.803041 21831 raft_consensus.cc:374] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.803083 21831 raft_consensus.cc:729] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.803210 21831 consensus_queue.cc:260] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.803642 21831 ts_tablet_manager.cc:1398] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:02.803828 21831 tablet_bootstrap.cc:492] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.804126 21834 tablet_bootstrap.cc:492] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.804302 21834 ts_tablet_manager.cc:1367] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20240501 19:56:02.804548 21831 tablet_bootstrap.cc:654] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.805495 21831 tablet_bootstrap.cc:492] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.805570 21831 ts_tablet_manager.cc:1367] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:56:02.805896 21831 raft_consensus.cc:348] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.805979 21831 raft_consensus.cc:374] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.806010 21831 raft_consensus.cc:729] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.806123 21831 consensus_queue.cc:260] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.806445 21831 ts_tablet_manager.cc:1398] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:02.806591 21831 tablet_bootstrap.cc:492] T db648e6633204661b39a21199c8786da P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.807736 21831 tablet_bootstrap.cc:654] T db648e6633204661b39a21199c8786da P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.808104 21834 raft_consensus.cc:348] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.808302 21834 raft_consensus.cc:374] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.808374 21834 raft_consensus.cc:729] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.808790 21831 tablet_bootstrap.cc:492] T db648e6633204661b39a21199c8786da P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.808854 21831 ts_tablet_manager.cc:1367] T db648e6633204661b39a21199c8786da P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 19:56:02.808779 21834 consensus_queue.cc:260] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.809134 21831 raft_consensus.cc:348] T db648e6633204661b39a21199c8786da P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.809204 21831 raft_consensus.cc:374] T db648e6633204661b39a21199c8786da P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.809227 21831 raft_consensus.cc:729] T db648e6633204661b39a21199c8786da P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.809315 21831 consensus_queue.cc:260] T db648e6633204661b39a21199c8786da P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.809530 21831 ts_tablet_manager.cc:1398] T db648e6633204661b39a21199c8786da P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.809696 21831 tablet_bootstrap.cc:492] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.810567 21831 tablet_bootstrap.cc:654] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.810587 21834 ts_tablet_manager.cc:1398] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20240501 19:56:02.810616 21833 tablet_bootstrap.cc:492] T bc0f6426eae845ec8c7ea96580449d59 P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.810685 21833 ts_tablet_manager.cc:1367] T bc0f6426eae845ec8c7ea96580449d59 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240501 19:56:02.810803 21834 tablet_bootstrap.cc:492] T f947ac041bd24301ba41e195afe282a7 P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.811125 21833 raft_consensus.cc:348] T bc0f6426eae845ec8c7ea96580449d59 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.811216 21833 raft_consensus.cc:374] T bc0f6426eae845ec8c7ea96580449d59 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.811256 21833 raft_consensus.cc:729] T bc0f6426eae845ec8c7ea96580449d59 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.811403 21833 consensus_queue.cc:260] T bc0f6426eae845ec8c7ea96580449d59 P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.811617 21834 tablet_bootstrap.cc:654] T f947ac041bd24301ba41e195afe282a7 P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.811707 21833 ts_tablet_manager.cc:1398] T bc0f6426eae845ec8c7ea96580449d59 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:02.811887 21833 tablet_bootstrap.cc:492] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.812791 21833 tablet_bootstrap.cc:654] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.813673 21831 tablet_bootstrap.cc:492] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.813762 21831 ts_tablet_manager.cc:1367] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:56:02.814317 21833 tablet_bootstrap.cc:492] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.814296 21831 raft_consensus.cc:348] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.814390 21833 ts_tablet_manager.cc:1367] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 19:56:02.814394 21831 raft_consensus.cc:374] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.814438 21831 raft_consensus.cc:729] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.814548 21831 consensus_queue.cc:260] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.814803 21831 ts_tablet_manager.cc:1398] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.814822 21833 raft_consensus.cc:348] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.814915 21833 raft_consensus.cc:374] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.814951 21833 raft_consensus.cc:729] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.815060 21834 tablet_bootstrap.cc:492] T f947ac041bd24301ba41e195afe282a7 P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.815088 21833 consensus_queue.cc:260] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.815141 21834 ts_tablet_manager.cc:1367] T f947ac041bd24301ba41e195afe282a7 P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240501 19:56:02.815356 21831 tablet_bootstrap.cc:492] T be54f71e60754ba88ab21bd3b92d6947 P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.815428 21833 ts_tablet_manager.cc:1398] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:02.815582 21833 tablet_bootstrap.cc:492] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.816203 21831 tablet_bootstrap.cc:654] T be54f71e60754ba88ab21bd3b92d6947 P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.816242 21834 raft_consensus.cc:348] T f947ac041bd24301ba41e195afe282a7 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.816341 21834 raft_consensus.cc:374] T f947ac041bd24301ba41e195afe282a7 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.816378 21834 raft_consensus.cc:729] T f947ac041bd24301ba41e195afe282a7 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.816529 21834 consensus_queue.cc:260] T f947ac041bd24301ba41e195afe282a7 P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.816686 21833 tablet_bootstrap.cc:654] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.817765 21833 tablet_bootstrap.cc:492] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.817842 21834 ts_tablet_manager.cc:1398] T f947ac041bd24301ba41e195afe282a7 P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240501 19:56:02.817857 21833 ts_tablet_manager.cc:1367] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 19:56:02.818006 21834 tablet_bootstrap.cc:492] T cc489e219ea8431eafa345e8a6417a9a P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.818275 21833 raft_consensus.cc:348] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.818380 21833 raft_consensus.cc:374] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.818418 21833 raft_consensus.cc:729] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.818534 21833 consensus_queue.cc:260] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.818783 21833 ts_tablet_manager.cc:1398] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.818923 21834 tablet_bootstrap.cc:654] T cc489e219ea8431eafa345e8a6417a9a P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.818933 21833 tablet_bootstrap.cc:492] T 1637a1003436499e9baa39df67ba7bca P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.819669 21833 tablet_bootstrap.cc:654] T 1637a1003436499e9baa39df67ba7bca P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.820392 21834 tablet_bootstrap.cc:492] T cc489e219ea8431eafa345e8a6417a9a P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.820659 21833 tablet_bootstrap.cc:492] T 1637a1003436499e9baa39df67ba7bca P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.820740 21834 ts_tablet_manager.cc:1367] T cc489e219ea8431eafa345e8a6417a9a P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 19:56:02.820744 21833 ts_tablet_manager.cc:1367] T 1637a1003436499e9baa39df67ba7bca P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:02.821082 21833 raft_consensus.cc:348] T 1637a1003436499e9baa39df67ba7bca P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.821161 21833 raft_consensus.cc:374] T 1637a1003436499e9baa39df67ba7bca P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.821197 21833 raft_consensus.cc:729] T 1637a1003436499e9baa39df67ba7bca P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.821298 21833 consensus_queue.cc:260] T 1637a1003436499e9baa39df67ba7bca P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.821305 21834 raft_consensus.cc:348] T cc489e219ea8431eafa345e8a6417a9a P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.821409 21834 raft_consensus.cc:374] T cc489e219ea8431eafa345e8a6417a9a P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.821452 21834 raft_consensus.cc:729] T cc489e219ea8431eafa345e8a6417a9a P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.821558 21833 ts_tablet_manager.cc:1398] T 1637a1003436499e9baa39df67ba7bca P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.821578 21834 consensus_queue.cc:260] T cc489e219ea8431eafa345e8a6417a9a P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.821707 21833 tablet_bootstrap.cc:492] T c829a89ebd3242ef9cacc7762ed72912 P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.821856 21834 ts_tablet_manager.cc:1398] T cc489e219ea8431eafa345e8a6417a9a P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.821990 21834 tablet_bootstrap.cc:492] T db648e6633204661b39a21199c8786da P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.822549 21833 tablet_bootstrap.cc:654] T c829a89ebd3242ef9cacc7762ed72912 P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.822857 21834 tablet_bootstrap.cc:654] T db648e6633204661b39a21199c8786da P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.823722 21831 tablet_bootstrap.cc:492] T be54f71e60754ba88ab21bd3b92d6947 P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.823824 21831 ts_tablet_manager.cc:1367] T be54f71e60754ba88ab21bd3b92d6947 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240501 19:56:02.823932 21834 tablet_bootstrap.cc:492] T db648e6633204661b39a21199c8786da P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.823993 21834 ts_tablet_manager.cc:1367] T db648e6633204661b39a21199c8786da P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:02.824213 21831 raft_consensus.cc:348] T be54f71e60754ba88ab21bd3b92d6947 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.824294 21831 raft_consensus.cc:374] T be54f71e60754ba88ab21bd3b92d6947 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.824326 21831 raft_consensus.cc:729] T be54f71e60754ba88ab21bd3b92d6947 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.824383 21834 raft_consensus.cc:348] T db648e6633204661b39a21199c8786da P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.824462 21834 raft_consensus.cc:374] T db648e6633204661b39a21199c8786da P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.824440 21831 consensus_queue.cc:260] T be54f71e60754ba88ab21bd3b92d6947 P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.824493 21834 raft_consensus.cc:729] T db648e6633204661b39a21199c8786da P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.824607 21834 consensus_queue.cc:260] T db648e6633204661b39a21199c8786da P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.824750 21831 ts_tablet_manager.cc:1398] T be54f71e60754ba88ab21bd3b92d6947 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.824831 21834 ts_tablet_manager.cc:1398] T db648e6633204661b39a21199c8786da P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.824903 21831 tablet_bootstrap.cc:492] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.825035 21834 tablet_bootstrap.cc:492] T f46458ee560a4bd8a1dfc050fe0a76f1 P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.825691 21831 tablet_bootstrap.cc:654] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.825896 21834 tablet_bootstrap.cc:654] T f46458ee560a4bd8a1dfc050fe0a76f1 P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.826689 21833 tablet_bootstrap.cc:492] T c829a89ebd3242ef9cacc7762ed72912 P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.826789 21833 ts_tablet_manager.cc:1367] T c829a89ebd3242ef9cacc7762ed72912 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 19:56:02.827204 21833 raft_consensus.cc:348] T c829a89ebd3242ef9cacc7762ed72912 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.827304 21833 raft_consensus.cc:374] T c829a89ebd3242ef9cacc7762ed72912 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.827339 21833 raft_consensus.cc:729] T c829a89ebd3242ef9cacc7762ed72912 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.827428 21834 tablet_bootstrap.cc:492] T f46458ee560a4bd8a1dfc050fe0a76f1 P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.827495 21834 ts_tablet_manager.cc:1367] T f46458ee560a4bd8a1dfc050fe0a76f1 P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 19:56:02.827443 21833 consensus_queue.cc:260] T c829a89ebd3242ef9cacc7762ed72912 P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.827690 21833 ts_tablet_manager.cc:1398] T c829a89ebd3242ef9cacc7762ed72912 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.827728 21831 tablet_bootstrap.cc:492] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.827806 21831 ts_tablet_manager.cc:1367] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:02.827857 21833 tablet_bootstrap.cc:492] T 1832a822dbe8451ea298baaa319b94e5 P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.827858 21834 raft_consensus.cc:348] T f46458ee560a4bd8a1dfc050fe0a76f1 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.827942 21834 raft_consensus.cc:374] T f46458ee560a4bd8a1dfc050fe0a76f1 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.827972 21834 raft_consensus.cc:729] T f46458ee560a4bd8a1dfc050fe0a76f1 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.828087 21834 consensus_queue.cc:260] T f46458ee560a4bd8a1dfc050fe0a76f1 P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.828182 21831 raft_consensus.cc:348] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.828265 21831 raft_consensus.cc:374] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.828337 21834 ts_tablet_manager.cc:1398] T f46458ee560a4bd8a1dfc050fe0a76f1 P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.828471 21834 tablet_bootstrap.cc:492] T be54f71e60754ba88ab21bd3b92d6947 P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.828308 21831 raft_consensus.cc:729] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.828641 21831 consensus_queue.cc:260] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.828689 21833 tablet_bootstrap.cc:654] T 1832a822dbe8451ea298baaa319b94e5 P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.829388 21834 tablet_bootstrap.cc:654] T be54f71e60754ba88ab21bd3b92d6947 P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.829685 21831 ts_tablet_manager.cc:1398] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:02.829890 21831 tablet_bootstrap.cc:492] T b1061bac480546159dce2050285db8b6 P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.830986 21831 tablet_bootstrap.cc:654] T b1061bac480546159dce2050285db8b6 P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.831056 21834 tablet_bootstrap.cc:492] T be54f71e60754ba88ab21bd3b92d6947 P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.831130 21834 ts_tablet_manager.cc:1367] T be54f71e60754ba88ab21bd3b92d6947 P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240501 19:56:02.831473 21834 raft_consensus.cc:348] T be54f71e60754ba88ab21bd3b92d6947 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.831558 21834 raft_consensus.cc:374] T be54f71e60754ba88ab21bd3b92d6947 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.831589 21834 raft_consensus.cc:729] T be54f71e60754ba88ab21bd3b92d6947 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.831709 21834 consensus_queue.cc:260] T be54f71e60754ba88ab21bd3b92d6947 P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.832185 21834 ts_tablet_manager.cc:1398] T be54f71e60754ba88ab21bd3b92d6947 P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.832389 21834 tablet_bootstrap.cc:492] T bc0f6426eae845ec8c7ea96580449d59 P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.833092 21833 tablet_bootstrap.cc:492] T 1832a822dbe8451ea298baaa319b94e5 P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.833180 21833 ts_tablet_manager.cc:1367] T 1832a822dbe8451ea298baaa319b94e5 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 19:56:02.833251 21831 tablet_bootstrap.cc:492] T b1061bac480546159dce2050285db8b6 P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.833319 21831 ts_tablet_manager.cc:1367] T b1061bac480546159dce2050285db8b6 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 19:56:02.833604 21834 tablet_bootstrap.cc:654] T bc0f6426eae845ec8c7ea96580449d59 P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.833586 21831 raft_consensus.cc:348] T b1061bac480546159dce2050285db8b6 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } }
I20240501 19:56:02.833647 21831 raft_consensus.cc:374] T b1061bac480546159dce2050285db8b6 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.833667 21831 raft_consensus.cc:729] T b1061bac480546159dce2050285db8b6 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.833614 21833 raft_consensus.cc:348] T 1832a822dbe8451ea298baaa319b94e5 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.833700 21833 raft_consensus.cc:374] T 1832a822dbe8451ea298baaa319b94e5 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.833738 21833 raft_consensus.cc:729] T 1832a822dbe8451ea298baaa319b94e5 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.833758 21831 consensus_queue.cc:260] T b1061bac480546159dce2050285db8b6 P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } }
I20240501 19:56:02.833884 21833 consensus_queue.cc:260] T 1832a822dbe8451ea298baaa319b94e5 P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.833997 21831 ts_tablet_manager.cc:1398] T b1061bac480546159dce2050285db8b6 P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.834161 21833 ts_tablet_manager.cc:1398] T 1832a822dbe8451ea298baaa319b94e5 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.834169 21831 tablet_bootstrap.cc:492] T 4516205f081c4ea59bd657d28f24ff9b P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.834947 21834 tablet_bootstrap.cc:492] T bc0f6426eae845ec8c7ea96580449d59 P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.835026 21834 ts_tablet_manager.cc:1367] T bc0f6426eae845ec8c7ea96580449d59 P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240501 19:56:02.835050 21831 tablet_bootstrap.cc:654] T 4516205f081c4ea59bd657d28f24ff9b P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.835424 21834 raft_consensus.cc:348] T bc0f6426eae845ec8c7ea96580449d59 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.835521 21834 raft_consensus.cc:374] T bc0f6426eae845ec8c7ea96580449d59 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.835552 21834 raft_consensus.cc:729] T bc0f6426eae845ec8c7ea96580449d59 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.835672 21834 consensus_queue.cc:260] T bc0f6426eae845ec8c7ea96580449d59 P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.835915 21834 ts_tablet_manager.cc:1398] T bc0f6426eae845ec8c7ea96580449d59 P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.835945 21833 tablet_bootstrap.cc:492] T 3c9a7537441d4deeae6178e0d06dc532 P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.836055 21834 tablet_bootstrap.cc:492] T 3c9a7537441d4deeae6178e0d06dc532 P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.836848 21833 tablet_bootstrap.cc:654] T 3c9a7537441d4deeae6178e0d06dc532 P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.837478 21834 tablet_bootstrap.cc:654] T 3c9a7537441d4deeae6178e0d06dc532 P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.837807 21833 tablet_bootstrap.cc:492] T 3c9a7537441d4deeae6178e0d06dc532 P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.837862 21833 ts_tablet_manager.cc:1367] T 3c9a7537441d4deeae6178e0d06dc532 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:02.838130 21833 raft_consensus.cc:348] T 3c9a7537441d4deeae6178e0d06dc532 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.838227 21833 raft_consensus.cc:374] T 3c9a7537441d4deeae6178e0d06dc532 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.838306 21833 raft_consensus.cc:729] T 3c9a7537441d4deeae6178e0d06dc532 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.838424 21833 consensus_queue.cc:260] T 3c9a7537441d4deeae6178e0d06dc532 P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.838682 21833 ts_tablet_manager.cc:1398] T 3c9a7537441d4deeae6178e0d06dc532 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.838724 21834 tablet_bootstrap.cc:492] T 3c9a7537441d4deeae6178e0d06dc532 P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.838778 21834 ts_tablet_manager.cc:1367] T 3c9a7537441d4deeae6178e0d06dc532 P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240501 19:56:02.838848 21833 tablet_bootstrap.cc:492] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.839056 21834 raft_consensus.cc:348] T 3c9a7537441d4deeae6178e0d06dc532 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.839115 21834 raft_consensus.cc:374] T 3c9a7537441d4deeae6178e0d06dc532 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.839134 21834 raft_consensus.cc:729] T 3c9a7537441d4deeae6178e0d06dc532 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.839200 21834 consensus_queue.cc:260] T 3c9a7537441d4deeae6178e0d06dc532 P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.839401 21834 ts_tablet_manager.cc:1398] T 3c9a7537441d4deeae6178e0d06dc532 P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.839509 21834 tablet_bootstrap.cc:492] T 1832a822dbe8451ea298baaa319b94e5 P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.839654 21833 tablet_bootstrap.cc:654] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.840195 21834 tablet_bootstrap.cc:654] T 1832a822dbe8451ea298baaa319b94e5 P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.841091 21834 tablet_bootstrap.cc:492] T 1832a822dbe8451ea298baaa319b94e5 P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.841153 21835 raft_consensus.cc:482] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:02.841179 21834 ts_tablet_manager.cc:1367] T 1832a822dbe8451ea298baaa319b94e5 P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 19:56:02.841296 21835 raft_consensus.cc:504] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.841562 21834 raft_consensus.cc:348] T 1832a822dbe8451ea298baaa319b94e5 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.841660 21834 raft_consensus.cc:374] T 1832a822dbe8451ea298baaa319b94e5 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.841689 21834 raft_consensus.cc:729] T 1832a822dbe8451ea298baaa319b94e5 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.841795 21834 consensus_queue.cc:260] T 1832a822dbe8451ea298baaa319b94e5 P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.842020 21834 ts_tablet_manager.cc:1398] T 1832a822dbe8451ea298baaa319b94e5 P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.842175 21834 tablet_bootstrap.cc:492] T 50ae742565b241da8372a1791ee7f1b4 P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.842676 21835 leader_election.cc:288] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.843142 21834 tablet_bootstrap.cc:654] T 50ae742565b241da8372a1791ee7f1b4 P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.844372 21834 tablet_bootstrap.cc:492] T 50ae742565b241da8372a1791ee7f1b4 P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.844455 21834 ts_tablet_manager.cc:1367] T 50ae742565b241da8372a1791ee7f1b4 P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:02.844859 21834 raft_consensus.cc:348] T 50ae742565b241da8372a1791ee7f1b4 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.844939 21834 raft_consensus.cc:374] T 50ae742565b241da8372a1791ee7f1b4 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.844960 21834 raft_consensus.cc:729] T 50ae742565b241da8372a1791ee7f1b4 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.845067 21834 consensus_queue.cc:260] T 50ae742565b241da8372a1791ee7f1b4 P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.845309 21834 ts_tablet_manager.cc:1398] T 50ae742565b241da8372a1791ee7f1b4 P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.845499 21834 tablet_bootstrap.cc:492] T a35bd26ec43741e08fc811450aef8ac6 P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.846846 21834 tablet_bootstrap.cc:654] T a35bd26ec43741e08fc811450aef8ac6 P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.847586 21831 tablet_bootstrap.cc:492] T 4516205f081c4ea59bd657d28f24ff9b P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.847681 21831 ts_tablet_manager.cc:1367] T 4516205f081c4ea59bd657d28f24ff9b P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240501 19:56:02.847748 21841 raft_consensus.cc:482] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:02.847887 21841 raft_consensus.cc:504] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.847994 21834 tablet_bootstrap.cc:492] T a35bd26ec43741e08fc811450aef8ac6 P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.848065 21834 ts_tablet_manager.cc:1367] T a35bd26ec43741e08fc811450aef8ac6 P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 19:56:02.848057 21831 raft_consensus.cc:348] T 4516205f081c4ea59bd657d28f24ff9b P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.848132 21831 raft_consensus.cc:374] T 4516205f081c4ea59bd657d28f24ff9b P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.848158 21831 raft_consensus.cc:729] T 4516205f081c4ea59bd657d28f24ff9b P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.848161 21505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1832a822dbe8451ea298baaa319b94e5" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" is_pre_election: true
I20240501 19:56:02.848300 21831 consensus_queue.cc:260] T 4516205f081c4ea59bd657d28f24ff9b P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.848326 21505 raft_consensus.cc:2454] T 1832a822dbe8451ea298baaa319b94e5 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 0.
I20240501 19:56:02.848424 21834 raft_consensus.cc:348] T a35bd26ec43741e08fc811450aef8ac6 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.848510 21834 raft_consensus.cc:374] T a35bd26ec43741e08fc811450aef8ac6 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.848521 21831 ts_tablet_manager.cc:1398] T 4516205f081c4ea59bd657d28f24ff9b P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.848552 21834 raft_consensus.cc:729] T a35bd26ec43741e08fc811450aef8ac6 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.848670 21831 tablet_bootstrap.cc:492] T 9aa9614626304947b240419d14d82fdb P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.848733 21569 leader_election.cc:302] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:02.848891 21841 leader_election.cc:288] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.848950 21834 consensus_queue.cc:260] T a35bd26ec43741e08fc811450aef8ac6 P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.849078 21838 raft_consensus.cc:2781] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:02.849138 21838 raft_consensus.cc:482] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:02.849186 21838 raft_consensus.cc:3037] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.849232 21834 ts_tablet_manager.cc:1398] T a35bd26ec43741e08fc811450aef8ac6 P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:02.849372 21834 tablet_bootstrap.cc:492] T c829a89ebd3242ef9cacc7762ed72912 P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.849543 21831 tablet_bootstrap.cc:654] T 9aa9614626304947b240419d14d82fdb P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.850188 21834 tablet_bootstrap.cc:654] T c829a89ebd3242ef9cacc7762ed72912 P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.850199 21838 raft_consensus.cc:504] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.850562 21838 leader_election.cc:288] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 election: Requested vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.850761 21833 tablet_bootstrap.cc:492] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.850850 21833 ts_tablet_manager.cc:1367] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240501 19:56:02.851269 21833 raft_consensus.cc:348] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } }
I20240501 19:56:02.851377 21833 raft_consensus.cc:374] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.851404 21833 raft_consensus.cc:729] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.851249 21505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1832a822dbe8451ea298baaa319b94e5" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0"
I20240501 19:56:02.851506 21833 consensus_queue.cc:260] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } }
I20240501 19:56:02.851588 21505 raft_consensus.cc:3037] T 1832a822dbe8451ea298baaa319b94e5 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.851773 21833 ts_tablet_manager.cc:1398] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.851912 21833 tablet_bootstrap.cc:492] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.852533 21505 raft_consensus.cc:2454] T 1832a822dbe8451ea298baaa319b94e5 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 1.
I20240501 19:56:02.852708 21833 tablet_bootstrap.cc:654] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.853035 21569 leader_election.cc:302] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:02.853607 21838 raft_consensus.cc:2781] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:02.853710 21838 raft_consensus.cc:686] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 LEADER]: Becoming Leader. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Running, Role: LEADER
I20240501 19:56:02.853989 21838 consensus_queue.cc:237] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.856606 21363 catalog_manager.cc:5484] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2). New cstate: current_term: 1 leader_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:02.857436 21831 tablet_bootstrap.cc:492] T 9aa9614626304947b240419d14d82fdb P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.857506 21831 ts_tablet_manager.cc:1367] T 9aa9614626304947b240419d14d82fdb P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240501 19:56:02.857873 21831 raft_consensus.cc:348] T 9aa9614626304947b240419d14d82fdb P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.857968 21831 raft_consensus.cc:374] T 9aa9614626304947b240419d14d82fdb P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.858006 21831 raft_consensus.cc:729] T 9aa9614626304947b240419d14d82fdb P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.858121 21831 consensus_queue.cc:260] T 9aa9614626304947b240419d14d82fdb P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.858147 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be54f71e60754ba88ab21bd3b92d6947" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:02.858395 21767 raft_consensus.cc:2454] T be54f71e60754ba88ab21bd3b92d6947 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:02.858479 21831 ts_tablet_manager.cc:1398] T 9aa9614626304947b240419d14d82fdb P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.858639 21831 tablet_bootstrap.cc:492] T e8ccd935e153446bae5651f2a7165a1b P 5bf436e65ed14967b1ef3ede9e48bebf: Bootstrap starting.
I20240501 19:56:02.859489 21831 tablet_bootstrap.cc:654] T e8ccd935e153446bae5651f2a7165a1b P 5bf436e65ed14967b1ef3ede9e48bebf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.859772 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be54f71e60754ba88ab21bd3b92d6947" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" is_pre_election: true
I20240501 19:56:02.859952 21636 raft_consensus.cc:2454] T be54f71e60754ba88ab21bd3b92d6947 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:02.860241 21439 leader_election.cc:302] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, aae2aec0505e468193ac56ee42f6211d; no voters:
I20240501 19:56:02.860481 21841 raft_consensus.cc:2781] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:02.860530 21841 raft_consensus.cc:482] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:02.860558 21831 tablet_bootstrap.cc:492] T e8ccd935e153446bae5651f2a7165a1b P 5bf436e65ed14967b1ef3ede9e48bebf: No bootstrap required, opened a new log
I20240501 19:56:02.860569 21841 raft_consensus.cc:3037] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.860627 21831 ts_tablet_manager.cc:1367] T e8ccd935e153446bae5651f2a7165a1b P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
W20240501 19:56:02.861282 21552 tablet.cc:2369] T 1637a1003436499e9baa39df67ba7bca P 5b2d64c1fbe5462baa64e103f2545ba0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:56:02.861518 21841 raft_consensus.cc:504] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.861631 21831 raft_consensus.cc:348] T e8ccd935e153446bae5651f2a7165a1b P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.861707 21831 raft_consensus.cc:374] T e8ccd935e153446bae5651f2a7165a1b P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.861737 21831 raft_consensus.cc:729] T e8ccd935e153446bae5651f2a7165a1b P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.861883 21841 leader_election.cc:288] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.861864 21831 consensus_queue.cc:260] T e8ccd935e153446bae5651f2a7165a1b P 5bf436e65ed14967b1ef3ede9e48bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.862146 21831 ts_tablet_manager.cc:1398] T e8ccd935e153446bae5651f2a7165a1b P 5bf436e65ed14967b1ef3ede9e48bebf: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 19:56:02.862650 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be54f71e60754ba88ab21bd3b92d6947" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
I20240501 19:56:02.862756 21636 raft_consensus.cc:3037] T be54f71e60754ba88ab21bd3b92d6947 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.862795 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be54f71e60754ba88ab21bd3b92d6947" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:02.862924 21767 raft_consensus.cc:3037] T be54f71e60754ba88ab21bd3b92d6947 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.863662 21636 raft_consensus.cc:2454] T be54f71e60754ba88ab21bd3b92d6947 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:02.863891 21767 raft_consensus.cc:2454] T be54f71e60754ba88ab21bd3b92d6947 P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:02.863971 21438 leader_election.cc:302] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:02.864156 21841 raft_consensus.cc:2781] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:02.864359 21841 raft_consensus.cc:686] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Running, Role: LEADER
I20240501 19:56:02.864521 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1832a822dbe8451ea298baaa319b94e5" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:02.864606 21766 raft_consensus.cc:3037] T 1832a822dbe8451ea298baaa319b94e5 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.864629 21841 consensus_queue.cc:237] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.865204 21766 raft_consensus.cc:2454] T 1832a822dbe8451ea298baaa319b94e5 P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 1.
I20240501 19:56:02.865358 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1832a822dbe8451ea298baaa319b94e5" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:02.865450 21767 raft_consensus.cc:2362] T 1832a822dbe8451ea298baaa319b94e5 P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 1. Re-sending same reply.
I20240501 19:56:02.866580 21834 tablet_bootstrap.cc:492] T c829a89ebd3242ef9cacc7762ed72912 P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.866668 21834 ts_tablet_manager.cc:1367] T c829a89ebd3242ef9cacc7762ed72912 P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240501 19:56:02.866739 21833 tablet_bootstrap.cc:492] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.866848 21833 ts_tablet_manager.cc:1367] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240501 19:56:02.867028 21834 raft_consensus.cc:348] T c829a89ebd3242ef9cacc7762ed72912 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.867106 21834 raft_consensus.cc:374] T c829a89ebd3242ef9cacc7762ed72912 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.867127 21834 raft_consensus.cc:729] T c829a89ebd3242ef9cacc7762ed72912 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.867220 21834 consensus_queue.cc:260] T c829a89ebd3242ef9cacc7762ed72912 P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.867233 21833 raft_consensus.cc:348] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.867322 21833 raft_consensus.cc:374] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.867365 21833 raft_consensus.cc:729] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.867457 21834 ts_tablet_manager.cc:1398] T c829a89ebd3242ef9cacc7762ed72912 P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.867476 21833 consensus_queue.cc:260] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.867568 21834 tablet_bootstrap.cc:492] T b1061bac480546159dce2050285db8b6 P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.867748 21833 ts_tablet_manager.cc:1398] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.867923 21833 tablet_bootstrap.cc:492] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.868301 21834 tablet_bootstrap.cc:654] T b1061bac480546159dce2050285db8b6 P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.868753 21833 tablet_bootstrap.cc:654] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.868863 21841 raft_consensus.cc:482] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:02.868955 21841 raft_consensus.cc:504] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } }
I20240501 19:56:02.869236 21841 leader_election.cc:288] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.869396 21834 tablet_bootstrap.cc:492] T b1061bac480546159dce2050285db8b6 P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.869475 21834 ts_tablet_manager.cc:1367] T b1061bac480546159dce2050285db8b6 P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:02.869518 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1061bac480546159dce2050285db8b6" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:02.869750 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1061bac480546159dce2050285db8b6" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" is_pre_election: true
I20240501 19:56:02.869863 21636 raft_consensus.cc:2454] T b1061bac480546159dce2050285db8b6 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:02.869900 21834 raft_consensus.cc:348] T b1061bac480546159dce2050285db8b6 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } }
I20240501 19:56:02.869980 21834 raft_consensus.cc:374] T b1061bac480546159dce2050285db8b6 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.870019 21834 raft_consensus.cc:729] T b1061bac480546159dce2050285db8b6 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.869913 21363 catalog_manager.cc:5484] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:02.870133 21438 leader_election.cc:302] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:02.870126 21834 consensus_queue.cc:260] T b1061bac480546159dce2050285db8b6 P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } }
I20240501 19:56:02.870292 21841 raft_consensus.cc:2781] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:02.870337 21841 raft_consensus.cc:482] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20240501 19:56:02.870316 21439 leader_election.cc:341] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:56:02.870374 21841 raft_consensus.cc:3037] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.870440 21834 ts_tablet_manager.cc:1398] T b1061bac480546159dce2050285db8b6 P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.870572 21834 tablet_bootstrap.cc:492] T 4516205f081c4ea59bd657d28f24ff9b P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.871055 21833 tablet_bootstrap.cc:492] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.871133 21833 ts_tablet_manager.cc:1367] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:02.871177 21841 raft_consensus.cc:504] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } }
I20240501 19:56:02.871454 21834 tablet_bootstrap.cc:654] T 4516205f081c4ea59bd657d28f24ff9b P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.871536 21841 leader_election.cc:288] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.871765 21833 raft_consensus.cc:348] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.871826 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1061bac480546159dce2050285db8b6" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:02.871868 21833 raft_consensus.cc:374] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.871899 21833 raft_consensus.cc:729] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.871904 21767 raft_consensus.cc:3037] T b1061bac480546159dce2050285db8b6 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.872015 21833 consensus_queue.cc:260] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.872411 21833 ts_tablet_manager.cc:1398] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.872438 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1061bac480546159dce2050285db8b6" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
I20240501 19:56:02.872516 21636 raft_consensus.cc:3037] T b1061bac480546159dce2050285db8b6 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.872570 21833 tablet_bootstrap.cc:492] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.872689 21767 raft_consensus.cc:2454] T b1061bac480546159dce2050285db8b6 P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:02.873091 21636 raft_consensus.cc:2454] T b1061bac480546159dce2050285db8b6 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:02.873332 21438 leader_election.cc:302] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:02.873410 21833 tablet_bootstrap.cc:654] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.873471 21841 raft_consensus.cc:2781] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:02.873541 21841 raft_consensus.cc:686] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Running, Role: LEADER
I20240501 19:56:02.873643 21841 consensus_queue.cc:237] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } }
I20240501 19:56:02.875290 21834 tablet_bootstrap.cc:492] T 4516205f081c4ea59bd657d28f24ff9b P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.875408 21834 ts_tablet_manager.cc:1367] T 4516205f081c4ea59bd657d28f24ff9b P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 19:56:02.875804 21834 raft_consensus.cc:348] T 4516205f081c4ea59bd657d28f24ff9b P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.875895 21834 raft_consensus.cc:374] T 4516205f081c4ea59bd657d28f24ff9b P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.875916 21834 raft_consensus.cc:729] T 4516205f081c4ea59bd657d28f24ff9b P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.876008 21834 consensus_queue.cc:260] T 4516205f081c4ea59bd657d28f24ff9b P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.876060 21361 catalog_manager.cc:5484] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: HEALTHY } } }
I20240501 19:56:02.876245 21834 ts_tablet_manager.cc:1398] T 4516205f081c4ea59bd657d28f24ff9b P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:02.876389 21834 tablet_bootstrap.cc:492] T e8ccd935e153446bae5651f2a7165a1b P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.877025 21834 tablet_bootstrap.cc:654] T e8ccd935e153446bae5651f2a7165a1b P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.878113 21833 tablet_bootstrap.cc:492] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.878222 21834 tablet_bootstrap.cc:492] T e8ccd935e153446bae5651f2a7165a1b P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.878301 21833 ts_tablet_manager.cc:1367] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 19:56:02.878326 21834 ts_tablet_manager.cc:1367] T e8ccd935e153446bae5651f2a7165a1b P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:02.878690 21833 raft_consensus.cc:348] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.878690 21834 raft_consensus.cc:348] T e8ccd935e153446bae5651f2a7165a1b P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.878788 21834 raft_consensus.cc:374] T e8ccd935e153446bae5651f2a7165a1b P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.878788 21833 raft_consensus.cc:374] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.878832 21833 raft_consensus.cc:729] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.878832 21834 raft_consensus.cc:729] T e8ccd935e153446bae5651f2a7165a1b P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.878942 21834 consensus_queue.cc:260] T e8ccd935e153446bae5651f2a7165a1b P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.878942 21833 consensus_queue.cc:260] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.879207 21834 ts_tablet_manager.cc:1398] T e8ccd935e153446bae5651f2a7165a1b P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.879364 21834 tablet_bootstrap.cc:492] T 9aa9614626304947b240419d14d82fdb P aae2aec0505e468193ac56ee42f6211d: Bootstrap starting.
I20240501 19:56:02.879580 21833 ts_tablet_manager.cc:1398] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.879750 21833 tablet_bootstrap.cc:492] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0: Bootstrap starting.
I20240501 19:56:02.880159 21834 tablet_bootstrap.cc:654] T 9aa9614626304947b240419d14d82fdb P aae2aec0505e468193ac56ee42f6211d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.881206 21833 tablet_bootstrap.cc:654] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:02.881712 21834 tablet_bootstrap.cc:492] T 9aa9614626304947b240419d14d82fdb P aae2aec0505e468193ac56ee42f6211d: No bootstrap required, opened a new log
I20240501 19:56:02.881788 21834 ts_tablet_manager.cc:1367] T 9aa9614626304947b240419d14d82fdb P aae2aec0505e468193ac56ee42f6211d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 19:56:02.882197 21834 raft_consensus.cc:348] T 9aa9614626304947b240419d14d82fdb P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.882328 21834 raft_consensus.cc:374] T 9aa9614626304947b240419d14d82fdb P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.882364 21834 raft_consensus.cc:729] T 9aa9614626304947b240419d14d82fdb P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.882392 21833 tablet_bootstrap.cc:492] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0: No bootstrap required, opened a new log
I20240501 19:56:02.882459 21833 ts_tablet_manager.cc:1367] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:02.882488 21834 consensus_queue.cc:260] T 9aa9614626304947b240419d14d82fdb P aae2aec0505e468193ac56ee42f6211d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.882771 21834 ts_tablet_manager.cc:1398] T 9aa9614626304947b240419d14d82fdb P aae2aec0505e468193ac56ee42f6211d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:02.882967 21833 raft_consensus.cc:348] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.883050 21833 raft_consensus.cc:374] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:02.883082 21833 raft_consensus.cc:729] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Initialized, Role: FOLLOWER
I20240501 19:56:02.883194 21833 consensus_queue.cc:260] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.883528 21833 ts_tablet_manager.cc:1398] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:02.893734 21838 raft_consensus.cc:482] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:02.893818 21838 raft_consensus.cc:504] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.894161 21838 leader_election.cc:288] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.894493 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc489e219ea8431eafa345e8a6417a9a" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:02.894495 21505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc489e219ea8431eafa345e8a6417a9a" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" is_pre_election: true
I20240501 19:56:02.894588 21505 raft_consensus.cc:2454] T cc489e219ea8431eafa345e8a6417a9a P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 0.
I20240501 19:56:02.894598 21767 raft_consensus.cc:2454] T cc489e219ea8431eafa345e8a6417a9a P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 0.
I20240501 19:56:02.894876 21570 leader_election.cc:302] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bf436e65ed14967b1ef3ede9e48bebf, aae2aec0505e468193ac56ee42f6211d; no voters:
I20240501 19:56:02.895054 21838 raft_consensus.cc:2781] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:02.895104 21838 raft_consensus.cc:482] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:02.895131 21838 raft_consensus.cc:3037] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.895840 21838 raft_consensus.cc:504] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.896133 21838 leader_election.cc:288] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 election: Requested vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.896427 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc489e219ea8431eafa345e8a6417a9a" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:02.896425 21505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc489e219ea8431eafa345e8a6417a9a" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0"
I20240501 19:56:02.896528 21767 raft_consensus.cc:3037] T cc489e219ea8431eafa345e8a6417a9a P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.896528 21505 raft_consensus.cc:3037] T cc489e219ea8431eafa345e8a6417a9a P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.896879 21841 raft_consensus.cc:482] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:02.896965 21841 raft_consensus.cc:504] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.897261 21841 leader_election.cc:288] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.897353 21767 raft_consensus.cc:2454] T cc489e219ea8431eafa345e8a6417a9a P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 1.
I20240501 19:56:02.897361 21505 raft_consensus.cc:2454] T cc489e219ea8431eafa345e8a6417a9a P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 1.
I20240501 19:56:02.897543 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa9614626304947b240419d14d82fdb" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" is_pre_election: true
I20240501 19:56:02.897650 21636 raft_consensus.cc:2454] T 9aa9614626304947b240419d14d82fdb P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:02.897691 21570 leader_election.cc:302] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bf436e65ed14967b1ef3ede9e48bebf, aae2aec0505e468193ac56ee42f6211d; no voters:
I20240501 19:56:02.897795 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa9614626304947b240419d14d82fdb" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:02.897864 21838 raft_consensus.cc:2781] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:02.897886 21767 raft_consensus.cc:2454] T 9aa9614626304947b240419d14d82fdb P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:02.897938 21838 raft_consensus.cc:686] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 LEADER]: Becoming Leader. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Running, Role: LEADER
I20240501 19:56:02.897929 21438 leader_election.cc:302] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:02.898083 21841 raft_consensus.cc:2781] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:02.898054 21838 consensus_queue.cc:237] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.898133 21841 raft_consensus.cc:482] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:02.898177 21841 raft_consensus.cc:3037] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.898964 21841 raft_consensus.cc:504] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.899307 21841 leader_election.cc:288] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.899525 21363 catalog_manager.cc:5484] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2). New cstate: current_term: 1 leader_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:02.899593 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa9614626304947b240419d14d82fdb" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:02.899664 21767 raft_consensus.cc:3037] T 9aa9614626304947b240419d14d82fdb P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.900115 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa9614626304947b240419d14d82fdb" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
I20240501 19:56:02.900216 21636 raft_consensus.cc:3037] T 9aa9614626304947b240419d14d82fdb P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.900317 21767 raft_consensus.cc:2454] T 9aa9614626304947b240419d14d82fdb P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:02.900573 21439 leader_election.cc:302] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, aae2aec0505e468193ac56ee42f6211d; no voters:
I20240501 19:56:02.900740 21841 raft_consensus.cc:2781] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:02.900807 21841 raft_consensus.cc:686] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Running, Role: LEADER
I20240501 19:56:02.900918 21841 consensus_queue.cc:237] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.901206 21636 raft_consensus.cc:2454] T 9aa9614626304947b240419d14d82fdb P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:02.902309 21361 catalog_manager.cc:5484] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:02.920683 21835 raft_consensus.cc:482] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:02.920760 21835 raft_consensus.cc:504] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.920982 21838 raft_consensus.cc:482] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:02.921089 21835 leader_election.cc:288] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.921065 21838 raft_consensus.cc:504] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.921370 21505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c829a89ebd3242ef9cacc7762ed72912" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" is_pre_election: true
I20240501 19:56:02.921420 21838 leader_election.cc:288] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.921489 21505 raft_consensus.cc:2454] T c829a89ebd3242ef9cacc7762ed72912 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 0.
I20240501 19:56:02.921516 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c829a89ebd3242ef9cacc7762ed72912" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:02.921622 21767 raft_consensus.cc:2454] T c829a89ebd3242ef9cacc7762ed72912 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 0.
I20240501 19:56:02.921799 21569 leader_election.cc:302] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:02.921856 21505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0f6426eae845ec8c7ea96580449d59" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" is_pre_election: true
I20240501 19:56:02.921912 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0f6426eae845ec8c7ea96580449d59" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:02.921953 21838 raft_consensus.cc:2781] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:02.921952 21505 raft_consensus.cc:2454] T bc0f6426eae845ec8c7ea96580449d59 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 0.
I20240501 19:56:02.921994 21838 raft_consensus.cc:482] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:02.922008 21767 raft_consensus.cc:2454] T bc0f6426eae845ec8c7ea96580449d59 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 0.
I20240501 19:56:02.922036 21838 raft_consensus.cc:3037] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.922259 21569 leader_election.cc:302] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:02.922407 21835 raft_consensus.cc:2781] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:02.922472 21835 raft_consensus.cc:482] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:02.922518 21835 raft_consensus.cc:3037] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.922784 21838 raft_consensus.cc:504] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.923087 21838 leader_election.cc:288] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 election: Requested vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.923254 21835 raft_consensus.cc:504] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.923354 21505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c829a89ebd3242ef9cacc7762ed72912" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0"
I20240501 19:56:02.923357 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c829a89ebd3242ef9cacc7762ed72912" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:02.923439 21505 raft_consensus.cc:3037] T c829a89ebd3242ef9cacc7762ed72912 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.923449 21767 raft_consensus.cc:3037] T c829a89ebd3242ef9cacc7762ed72912 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.923599 21835 leader_election.cc:288] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 election: Requested vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.924095 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0f6426eae845ec8c7ea96580449d59" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:02.924125 21504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0f6426eae845ec8c7ea96580449d59" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0"
I20240501 19:56:02.924199 21766 raft_consensus.cc:3037] T bc0f6426eae845ec8c7ea96580449d59 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.924216 21767 raft_consensus.cc:2454] T c829a89ebd3242ef9cacc7762ed72912 P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 1.
I20240501 19:56:02.924227 21504 raft_consensus.cc:3037] T bc0f6426eae845ec8c7ea96580449d59 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.924273 21505 raft_consensus.cc:2454] T c829a89ebd3242ef9cacc7762ed72912 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 1.
I20240501 19:56:02.924476 21570 leader_election.cc:302] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bf436e65ed14967b1ef3ede9e48bebf, aae2aec0505e468193ac56ee42f6211d; no voters:
I20240501 19:56:02.924640 21835 raft_consensus.cc:2781] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:02.924717 21835 raft_consensus.cc:686] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 LEADER]: Becoming Leader. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Running, Role: LEADER
I20240501 19:56:02.924835 21835 consensus_queue.cc:237] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.925001 21766 raft_consensus.cc:2454] T bc0f6426eae845ec8c7ea96580449d59 P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 1.
I20240501 19:56:02.925130 21504 raft_consensus.cc:2454] T bc0f6426eae845ec8c7ea96580449d59 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 1.
I20240501 19:56:02.925362 21569 leader_election.cc:302] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:02.925529 21835 raft_consensus.cc:2781] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:02.925611 21835 raft_consensus.cc:686] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 LEADER]: Becoming Leader. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Running, Role: LEADER
I20240501 19:56:02.925724 21835 consensus_queue.cc:237] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.926324 21361 catalog_manager.cc:5484] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2). New cstate: current_term: 1 leader_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:02.928825 21361 catalog_manager.cc:5484] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2). New cstate: current_term: 1 leader_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:02.941567 21841 raft_consensus.cc:482] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:02.941651 21841 raft_consensus.cc:504] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.941948 21841 leader_election.cc:288] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.942224 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f46458ee560a4bd8a1dfc050fe0a76f1" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" is_pre_election: true
I20240501 19:56:02.942315 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f46458ee560a4bd8a1dfc050fe0a76f1" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:02.942368 21636 raft_consensus.cc:2454] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:02.942420 21766 raft_consensus.cc:2454] T f46458ee560a4bd8a1dfc050fe0a76f1 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:02.942631 21438 leader_election.cc:302] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:02.942790 21841 raft_consensus.cc:2781] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:02.942854 21841 raft_consensus.cc:482] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:02.942904 21841 raft_consensus.cc:3037] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.943516 21841 raft_consensus.cc:504] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.943809 21841 leader_election.cc:288] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.944092 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f46458ee560a4bd8a1dfc050fe0a76f1" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
I20240501 19:56:02.944120 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f46458ee560a4bd8a1dfc050fe0a76f1" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:02.944190 21636 raft_consensus.cc:3037] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.944201 21766 raft_consensus.cc:3037] T f46458ee560a4bd8a1dfc050fe0a76f1 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.944952 21766 raft_consensus.cc:2454] T f46458ee560a4bd8a1dfc050fe0a76f1 P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:02.944952 21636 raft_consensus.cc:2454] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:02.945199 21439 leader_election.cc:302] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, aae2aec0505e468193ac56ee42f6211d; no voters:
I20240501 19:56:02.945367 21841 raft_consensus.cc:2781] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:02.945468 21841 raft_consensus.cc:686] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Running, Role: LEADER
I20240501 19:56:02.945583 21841 consensus_queue.cc:237] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.946980 21361 catalog_manager.cc:5484] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
W20240501 19:56:02.948338 21814 tablet.cc:2369] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:56:02.961076 21841 raft_consensus.cc:482] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:02.961177 21841 raft_consensus.cc:504] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.961477 21841 leader_election.cc:288] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.961717 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35bd26ec43741e08fc811450aef8ac6" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" is_pre_election: true
I20240501 19:56:02.961760 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35bd26ec43741e08fc811450aef8ac6" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:02.961855 21766 raft_consensus.cc:2454] T a35bd26ec43741e08fc811450aef8ac6 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:02.961850 21636 raft_consensus.cc:2454] T a35bd26ec43741e08fc811450aef8ac6 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:02.962064 21439 leader_election.cc:302] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, aae2aec0505e468193ac56ee42f6211d; no voters:
I20240501 19:56:02.962219 21841 raft_consensus.cc:2781] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:02.962323 21841 raft_consensus.cc:482] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:02.962363 21841 raft_consensus.cc:3037] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.963047 21841 raft_consensus.cc:504] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.963362 21841 leader_election.cc:288] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.963629 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35bd26ec43741e08fc811450aef8ac6" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:02.963639 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35bd26ec43741e08fc811450aef8ac6" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
I20240501 19:56:02.963729 21636 raft_consensus.cc:3037] T a35bd26ec43741e08fc811450aef8ac6 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.963730 21766 raft_consensus.cc:3037] T a35bd26ec43741e08fc811450aef8ac6 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.964430 21636 raft_consensus.cc:2454] T a35bd26ec43741e08fc811450aef8ac6 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:02.964563 21766 raft_consensus.cc:2454] T a35bd26ec43741e08fc811450aef8ac6 P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:02.964821 21439 leader_election.cc:302] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, aae2aec0505e468193ac56ee42f6211d; no voters:
I20240501 19:56:02.964977 21841 raft_consensus.cc:2781] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:02.965061 21841 raft_consensus.cc:686] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Running, Role: LEADER
I20240501 19:56:02.965174 21841 consensus_queue.cc:237] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.966555 21361 catalog_manager.cc:5484] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:02.968861 21835 raft_consensus.cc:482] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:02.968926 21835 raft_consensus.cc:504] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.969205 21835 leader_election.cc:288] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.969476 21504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9a7537441d4deeae6178e0d06dc532" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" is_pre_election: true
I20240501 19:56:02.969484 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9a7537441d4deeae6178e0d06dc532" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:02.969565 21504 raft_consensus.cc:2454] T 3c9a7537441d4deeae6178e0d06dc532 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 0.
I20240501 19:56:02.969594 21766 raft_consensus.cc:2454] T 3c9a7537441d4deeae6178e0d06dc532 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 0.
I20240501 19:56:02.969786 21569 leader_election.cc:302] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:02.969926 21835 raft_consensus.cc:2781] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:02.969982 21835 raft_consensus.cc:482] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:02.970017 21835 raft_consensus.cc:3037] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.970798 21835 raft_consensus.cc:504] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.971132 21835 leader_election.cc:288] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 election: Requested vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.971418 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9a7537441d4deeae6178e0d06dc532" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:02.971415 21504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c9a7537441d4deeae6178e0d06dc532" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0"
I20240501 19:56:02.971516 21766 raft_consensus.cc:3037] T 3c9a7537441d4deeae6178e0d06dc532 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.971516 21504 raft_consensus.cc:3037] T 3c9a7537441d4deeae6178e0d06dc532 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.972282 21766 raft_consensus.cc:2454] T 3c9a7537441d4deeae6178e0d06dc532 P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 1.
I20240501 19:56:02.972322 21504 raft_consensus.cc:2454] T 3c9a7537441d4deeae6178e0d06dc532 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 1.
I20240501 19:56:02.972554 21570 leader_election.cc:302] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bf436e65ed14967b1ef3ede9e48bebf, aae2aec0505e468193ac56ee42f6211d; no voters:
I20240501 19:56:02.972699 21835 raft_consensus.cc:2781] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:02.972777 21835 raft_consensus.cc:686] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 LEADER]: Becoming Leader. State: Replica: 5bf436e65ed14967b1ef3ede9e48bebf, State: Running, Role: LEADER
I20240501 19:56:02.972890 21835 consensus_queue.cc:237] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.974217 21361 catalog_manager.cc:5484] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2). New cstate: current_term: 1 leader_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:02.990378 21841 raft_consensus.cc:482] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:02.990542 21841 raft_consensus.cc:504] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.991194 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db648e6633204661b39a21199c8786da" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" is_pre_election: true
I20240501 19:56:02.991220 21841 leader_election.cc:288] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.991307 21636 raft_consensus.cc:2454] T db648e6633204661b39a21199c8786da P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:02.991551 21438 leader_election.cc:302] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:02.991593 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db648e6633204661b39a21199c8786da" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:02.991700 21766 raft_consensus.cc:2454] T db648e6633204661b39a21199c8786da P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:02.991703 21841 raft_consensus.cc:2781] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:02.991755 21841 raft_consensus.cc:482] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:02.991791 21841 raft_consensus.cc:3037] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.992609 21841 raft_consensus.cc:504] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.993144 21841 leader_election.cc:288] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:02.993186 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db648e6633204661b39a21199c8786da" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
I20240501 19:56:02.993271 21636 raft_consensus.cc:3037] T db648e6633204661b39a21199c8786da P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.993336 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db648e6633204661b39a21199c8786da" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:02.993435 21766 raft_consensus.cc:3037] T db648e6633204661b39a21199c8786da P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:02.994016 21636 raft_consensus.cc:2454] T db648e6633204661b39a21199c8786da P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:02.994292 21438 leader_election.cc:302] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:02.994297 21766 raft_consensus.cc:2454] T db648e6633204661b39a21199c8786da P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:02.994422 21841 raft_consensus.cc:2781] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:02.994758 21841 raft_consensus.cc:686] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Running, Role: LEADER
I20240501 19:56:02.994982 21841 consensus_queue.cc:237] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:02.996544 21361 catalog_manager.cc:5484] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:03.012760 21841 raft_consensus.cc:482] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:03.012866 21841 raft_consensus.cc:504] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.012965 21835 raft_consensus.cc:482] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:03.013054 21835 raft_consensus.cc:504] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.013195 21841 leader_election.cc:288] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:03.013388 21835 leader_election.cc:288] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:03.013481 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f947ac041bd24301ba41e195afe282a7" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:03.013485 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f947ac041bd24301ba41e195afe282a7" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" is_pre_election: true
I20240501 19:56:03.013583 21766 raft_consensus.cc:2454] T f947ac041bd24301ba41e195afe282a7 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:03.013597 21636 raft_consensus.cc:2454] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:03.013635 21504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f947ac041bd24301ba41e195afe282a7" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" is_pre_election: true
I20240501 19:56:03.013725 21504 raft_consensus.cc:2454] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 0.
I20240501 19:56:03.013720 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f947ac041bd24301ba41e195afe282a7" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:03.013799 21767 raft_consensus.cc:2454] T f947ac041bd24301ba41e195afe282a7 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bf436e65ed14967b1ef3ede9e48bebf in term 0.
I20240501 19:56:03.013906 21438 leader_election.cc:302] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:03.013989 21569 leader_election.cc:302] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:03.014117 21841 raft_consensus.cc:2781] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:03.014149 21835 raft_consensus.cc:2781] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:03.014174 21841 raft_consensus.cc:482] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:03.014191 21835 raft_consensus.cc:482] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:03.014214 21841 raft_consensus.cc:3037] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.014261 21835 raft_consensus.cc:3037] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.014848 21841 raft_consensus.cc:504] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.014871 21835 raft_consensus.cc:504] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.015154 21841 leader_election.cc:288] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:03.015169 21835 leader_election.cc:288] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 election: Requested vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:03.015465 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f947ac041bd24301ba41e195afe282a7" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
I20240501 19:56:03.015465 21504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f947ac041bd24301ba41e195afe282a7" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0"
I20240501 19:56:03.015528 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f947ac041bd24301ba41e195afe282a7" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:03.015610 21504 raft_consensus.cc:2379] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5bf436e65ed14967b1ef3ede9e48bebf in current term 1: Already voted for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in this term.
I20240501 19:56:03.015609 21636 raft_consensus.cc:2379] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b2d64c1fbe5462baa64e103f2545ba0 in current term 1: Already voted for candidate 5bf436e65ed14967b1ef3ede9e48bebf in this term.
I20240501 19:56:03.015619 21766 raft_consensus.cc:3037] T f947ac041bd24301ba41e195afe282a7 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.015858 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f947ac041bd24301ba41e195afe282a7" candidate_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:03.016240 21570 leader_election.cc:302] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5bf436e65ed14967b1ef3ede9e48bebf; no voters: 5b2d64c1fbe5462baa64e103f2545ba0, aae2aec0505e468193ac56ee42f6211d
I20240501 19:56:03.016378 21766 raft_consensus.cc:2454] T f947ac041bd24301ba41e195afe282a7 P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:03.016410 21835 raft_consensus.cc:2726] T f947ac041bd24301ba41e195afe282a7 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240501 19:56:03.016616 21439 leader_election.cc:302] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, aae2aec0505e468193ac56ee42f6211d; no voters: 5bf436e65ed14967b1ef3ede9e48bebf
I20240501 19:56:03.016757 21841 raft_consensus.cc:2781] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:03.016845 21841 raft_consensus.cc:686] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Running, Role: LEADER
I20240501 19:56:03.016952 21841 consensus_queue.cc:237] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.018333 21361 catalog_manager.cc:5484] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:03.024015 21841 raft_consensus.cc:482] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:03.024089 21841 raft_consensus.cc:504] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.024379 21841 leader_election.cc:288] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:03.024617 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50ae742565b241da8372a1791ee7f1b4" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" is_pre_election: true
I20240501 19:56:03.024638 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50ae742565b241da8372a1791ee7f1b4" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:03.024703 21636 raft_consensus.cc:2454] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:03.024726 21766 raft_consensus.cc:2454] T 50ae742565b241da8372a1791ee7f1b4 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:03.024941 21438 leader_election.cc:302] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:03.025102 21841 raft_consensus.cc:2781] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:03.025156 21841 raft_consensus.cc:482] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:03.025187 21841 raft_consensus.cc:3037] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.025727 21841 raft_consensus.cc:504] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.025981 21841 leader_election.cc:288] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:03.026217 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50ae742565b241da8372a1791ee7f1b4" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
I20240501 19:56:03.026288 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50ae742565b241da8372a1791ee7f1b4" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:03.026340 21636 raft_consensus.cc:3037] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.026391 21766 raft_consensus.cc:3037] T 50ae742565b241da8372a1791ee7f1b4 P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.026902 21766 raft_consensus.cc:2454] T 50ae742565b241da8372a1791ee7f1b4 P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:03.026902 21636 raft_consensus.cc:2454] T 50ae742565b241da8372a1791ee7f1b4 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:03.027132 21439 leader_election.cc:302] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, aae2aec0505e468193ac56ee42f6211d; no voters:
I20240501 19:56:03.027319 21841 raft_consensus.cc:2781] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:03.027383 21841 raft_consensus.cc:686] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Running, Role: LEADER
I20240501 19:56:03.027508 21841 consensus_queue.cc:237] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.028800 21361 catalog_manager.cc:5484] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:03.080158 21841 raft_consensus.cc:482] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:03.080302 21841 raft_consensus.cc:504] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.080698 21841 leader_election.cc:288] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:03.081058 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4516205f081c4ea59bd657d28f24ff9b" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" is_pre_election: true
I20240501 19:56:03.081064 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4516205f081c4ea59bd657d28f24ff9b" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:03.081176 21766 raft_consensus.cc:2454] T 4516205f081c4ea59bd657d28f24ff9b P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:03.081188 21636 raft_consensus.cc:2454] T 4516205f081c4ea59bd657d28f24ff9b P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:03.081473 21438 leader_election.cc:302] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:03.081629 21841 raft_consensus.cc:2781] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:03.081689 21841 raft_consensus.cc:482] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:03.081733 21841 raft_consensus.cc:3037] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.082453 21841 raft_consensus.cc:504] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.082721 21841 leader_election.cc:288] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:03.083011 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4516205f081c4ea59bd657d28f24ff9b" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
I20240501 19:56:03.083024 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4516205f081c4ea59bd657d28f24ff9b" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:03.083101 21636 raft_consensus.cc:3037] T 4516205f081c4ea59bd657d28f24ff9b P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.083135 21766 raft_consensus.cc:3037] T 4516205f081c4ea59bd657d28f24ff9b P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.083712 21636 raft_consensus.cc:2454] T 4516205f081c4ea59bd657d28f24ff9b P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:03.083765 21766 raft_consensus.cc:2454] T 4516205f081c4ea59bd657d28f24ff9b P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:03.084033 21438 leader_election.cc:302] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:03.084163 21841 raft_consensus.cc:2781] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:03.084246 21841 raft_consensus.cc:686] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Running, Role: LEADER
I20240501 19:56:03.084386 21841 consensus_queue.cc:237] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.085786 21361 catalog_manager.cc:5484] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:03.120324 21852 raft_consensus.cc:482] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:03.120435 21852 raft_consensus.cc:504] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.121471 21852 leader_election.cc:288] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565)
I20240501 19:56:03.125059 21504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1637a1003436499e9baa39df67ba7bca" candidate_uuid: "aae2aec0505e468193ac56ee42f6211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" is_pre_election: true
I20240501 19:56:03.125200 21504 raft_consensus.cc:2454] T 1637a1003436499e9baa39df67ba7bca P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aae2aec0505e468193ac56ee42f6211d in term 0.
I20240501 19:56:03.125387 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1637a1003436499e9baa39df67ba7bca" candidate_uuid: "aae2aec0505e468193ac56ee42f6211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" is_pre_election: true
I20240501 19:56:03.125506 21636 raft_consensus.cc:2454] T 1637a1003436499e9baa39df67ba7bca P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aae2aec0505e468193ac56ee42f6211d in term 0.
I20240501 19:56:03.125530 21700 leader_election.cc:302] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, aae2aec0505e468193ac56ee42f6211d; no voters:
I20240501 19:56:03.125749 21852 raft_consensus.cc:2781] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:03.125804 21852 raft_consensus.cc:482] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:03.125834 21852 raft_consensus.cc:3037] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.126605 21852 raft_consensus.cc:504] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.126879 21852 leader_election.cc:288] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [CANDIDATE]: Term 1 election: Requested vote from peers 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1:33105), 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565)
I20240501 19:56:03.127172 21504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1637a1003436499e9baa39df67ba7bca" candidate_uuid: "aae2aec0505e468193ac56ee42f6211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b2d64c1fbe5462baa64e103f2545ba0"
I20240501 19:56:03.127223 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1637a1003436499e9baa39df67ba7bca" candidate_uuid: "aae2aec0505e468193ac56ee42f6211d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
I20240501 19:56:03.127297 21504 raft_consensus.cc:3037] T 1637a1003436499e9baa39df67ba7bca P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.127302 21636 raft_consensus.cc:3037] T 1637a1003436499e9baa39df67ba7bca P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.127940 21636 raft_consensus.cc:2454] T 1637a1003436499e9baa39df67ba7bca P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aae2aec0505e468193ac56ee42f6211d in term 1.
I20240501 19:56:03.127979 21504 raft_consensus.cc:2454] T 1637a1003436499e9baa39df67ba7bca P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aae2aec0505e468193ac56ee42f6211d in term 1.
I20240501 19:56:03.128186 21700 leader_election.cc:302] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bf436e65ed14967b1ef3ede9e48bebf, aae2aec0505e468193ac56ee42f6211d; no voters:
I20240501 19:56:03.128337 21852 raft_consensus.cc:2781] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:03.128532 21852 raft_consensus.cc:686] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [term 1 LEADER]: Becoming Leader. State: Replica: aae2aec0505e468193ac56ee42f6211d, State: Running, Role: LEADER
I20240501 19:56:03.128816 21852 consensus_queue.cc:237] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.131619 21361 catalog_manager.cc:5484] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d reported cstate change: term changed from 0 to 1, leader changed from <none> to aae2aec0505e468193ac56ee42f6211d (127.20.210.3). New cstate: current_term: 1 leader_uuid: "aae2aec0505e468193ac56ee42f6211d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: HEALTHY } } }
I20240501 19:56:03.233575 21841 raft_consensus.cc:482] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:03.233698 21841 raft_consensus.cc:504] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.234056 21841 leader_election.cc:288] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:03.234416 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ccd935e153446bae5651f2a7165a1b" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" is_pre_election: true
I20240501 19:56:03.234478 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ccd935e153446bae5651f2a7165a1b" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d" is_pre_election: true
I20240501 19:56:03.234547 21636 raft_consensus.cc:2454] T e8ccd935e153446bae5651f2a7165a1b P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:03.234601 21766 raft_consensus.cc:2454] T e8ccd935e153446bae5651f2a7165a1b P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 0.
I20240501 19:56:03.234859 21438 leader_election.cc:302] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:03.235003 21841 raft_consensus.cc:2781] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:03.235052 21841 raft_consensus.cc:482] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:03.235098 21841 raft_consensus.cc:3037] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.237694 21841 raft_consensus.cc:504] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.238016 21841 leader_election.cc:288] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 5bf436e65ed14967b1ef3ede9e48bebf (127.20.210.2:35565), aae2aec0505e468193ac56ee42f6211d (127.20.210.3:37115)
I20240501 19:56:03.238373 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ccd935e153446bae5651f2a7165a1b" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf436e65ed14967b1ef3ede9e48bebf"
I20240501 19:56:03.238373 21766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ccd935e153446bae5651f2a7165a1b" candidate_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae2aec0505e468193ac56ee42f6211d"
I20240501 19:56:03.238493 21636 raft_consensus.cc:3037] T e8ccd935e153446bae5651f2a7165a1b P 5bf436e65ed14967b1ef3ede9e48bebf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.238493 21766 raft_consensus.cc:3037] T e8ccd935e153446bae5651f2a7165a1b P aae2aec0505e468193ac56ee42f6211d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:03.239359 21636 raft_consensus.cc:2454] T e8ccd935e153446bae5651f2a7165a1b P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:03.239354 21766 raft_consensus.cc:2454] T e8ccd935e153446bae5651f2a7165a1b P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b2d64c1fbe5462baa64e103f2545ba0 in term 1.
I20240501 19:56:03.239617 21438 leader_election.cc:302] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b2d64c1fbe5462baa64e103f2545ba0, 5bf436e65ed14967b1ef3ede9e48bebf; no voters:
I20240501 19:56:03.239794 21841 raft_consensus.cc:2781] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:03.239877 21841 raft_consensus.cc:686] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 5b2d64c1fbe5462baa64e103f2545ba0, State: Running, Role: LEADER
I20240501 19:56:03.239998 21841 consensus_queue.cc:237] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } }
I20240501 19:56:03.241489 21361 catalog_manager.cc:5484] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b2d64c1fbe5462baa64e103f2545ba0 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:03.247614 21841 consensus_queue.cc:1035] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.251413 21914 consensus_queue.cc:1035] T be54f71e60754ba88ab21bd3b92d6947 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.278187 21914 consensus_queue.cc:1035] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.280715 21841 consensus_queue.cc:1035] T b1061bac480546159dce2050285db8b6 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.354208 21841 consensus_queue.cc:1035] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.357141 21841 consensus_queue.cc:1035] T 9aa9614626304947b240419d14d82fdb P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.381646 21835 consensus_queue.cc:1035] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.385450 21838 consensus_queue.cc:1035] T 1832a822dbe8451ea298baaa319b94e5 P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.406879 21838 consensus_queue.cc:1035] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.409742 21835 consensus_queue.cc:1035] T 3c9a7537441d4deeae6178e0d06dc532 P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.435446 21841 consensus_queue.cc:1035] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.438613 21885 consensus_queue.cc:1035] T 50ae742565b241da8372a1791ee7f1b4 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.438660 21841 consensus_queue.cc:1035] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.440656 21841 consensus_queue.cc:1035] T a35bd26ec43741e08fc811450aef8ac6 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 19:56:03.450027 21919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:03.450189 21919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:03.452946 21919 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 19:56:03.461793 21835 consensus_queue.cc:1035] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.467023 21841 consensus_queue.cc:1035] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.467303 21835 consensus_queue.cc:1035] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.469859 21841 consensus_queue.cc:1035] T f947ac041bd24301ba41e195afe282a7 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.470738 21838 consensus_queue.cc:1035] T cc489e219ea8431eafa345e8a6417a9a P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_0'
I20240501 19:56:03.492102 21923 consensus_queue.cc:1035] T bc0f6426eae845ec8c7ea96580449d59 P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.518747 21835 consensus_queue.cc:1035] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.526469 21838 consensus_queue.cc:1035] T c829a89ebd3242ef9cacc7762ed72912 P 5bf436e65ed14967b1ef3ede9e48bebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.533244 21635 raft_consensus.cc:1261] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Refusing update from remote peer 5b2d64c1fbe5462baa64e103f2545ba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:03.533736 21914 consensus_queue.cc:1035] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.538065 21632 raft_consensus.cc:1261] T e8ccd935e153446bae5651f2a7165a1b P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Refusing update from remote peer 5b2d64c1fbe5462baa64e103f2545ba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:03.538388 21631 raft_consensus.cc:1261] T db648e6633204661b39a21199c8786da P 5bf436e65ed14967b1ef3ede9e48bebf [term 1 FOLLOWER]: Refusing update from remote peer 5b2d64c1fbe5462baa64e103f2545ba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:03.539652 21760 raft_consensus.cc:1261] T db648e6633204661b39a21199c8786da P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Refusing update from remote peer 5b2d64c1fbe5462baa64e103f2545ba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:03.539708 21763 raft_consensus.cc:1261] T f46458ee560a4bd8a1dfc050fe0a76f1 P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Refusing update from remote peer 5b2d64c1fbe5462baa64e103f2545ba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:03.541148 21766 raft_consensus.cc:1261] T e8ccd935e153446bae5651f2a7165a1b P aae2aec0505e468193ac56ee42f6211d [term 1 FOLLOWER]: Refusing update from remote peer 5b2d64c1fbe5462baa64e103f2545ba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:03.542660 21885 consensus_queue.cc:1035] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.542660 21841 consensus_queue.cc:1035] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.546542 21841 consensus_queue.cc:1035] T f46458ee560a4bd8a1dfc050fe0a76f1 P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.546782 21841 consensus_queue.cc:1035] T db648e6633204661b39a21199c8786da P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.547021 21841 consensus_queue.cc:1035] T e8ccd935e153446bae5651f2a7165a1b P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.558785 21971 mvcc.cc:205] Tried to move back new op lower bound from 7022974417028636672 to 7022974414626234368. Current Snapshot: MvccSnapshot[applied={T|T < 7022974417028636672}]
I20240501 19:56:03.566067 21974 mvcc.cc:205] Tried to move back new op lower bound from 7022974417028636672 to 7022974414626234368. Current Snapshot: MvccSnapshot[applied={T|T < 7022974417028636672}]
I20240501 19:56:03.568611 21908 consensus_queue.cc:1035] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b2d64c1fbe5462baa64e103f2545ba0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.580332 21852 consensus_queue.cc:1035] T 1637a1003436499e9baa39df67ba7bca P aae2aec0505e468193ac56ee42f6211d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 109.247 ms
time per row: 0.0546235 ms
I20240501 19:56:03.602596 21983 consensus_queue.cc:1035] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae2aec0505e468193ac56ee42f6211d" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.608263 22003 mvcc.cc:205] Tried to move back new op lower bound from 7022974417065967616 to 7022974414626234368. Current Snapshot: MvccSnapshot[applied={T|T < 7022974417028636672}]
I20240501 19:56:03.612700 21990 consensus_queue.cc:1035] T 4516205f081c4ea59bd657d28f24ff9b P 5b2d64c1fbe5462baa64e103f2545ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bf436e65ed14967b1ef3ede9e48bebf" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:03.792884 21485 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240501 19:56:03.793511 21747 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:03.794932 21616 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:56:03.884884 21747 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:03.885011 21485 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240501 19:56:03.885573 21616 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:56:03.979694 21616 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:56:03.980232 21485 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240501 19:56:03.992321 21747 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:04.198266 21616 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:56:04.198654 21485 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240501 19:56:04.198768 21747 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:04.283155 21485 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240501 19:56:04.283722 21616 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:56:04.284274 21747 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:04.314085 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 21423
I20240501 19:56:04.319792 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 21554
I20240501 19:56:04.325502 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 21685
I20240501 19:56:04.330833 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 21332
2024-05-01T19:56:04Z chronyd exiting
[ OK ] TableRangeRebalancingTest.Basic (6139 ms)
[----------] 1 test from TableRangeRebalancingTest (6139 ms total)
[----------] 1 test from RebalanceStartSafetyTest
[ RUN ] RebalanceStartSafetyTest.TooManyIgnoredTservers/1
I20240501 19:56:04.358515 21320 test_util.cc:268] Using random seed: -529877650
I20240501 19:56:04.359129 21320 ts_itest-base.cc:112] Starting cluster with:
I20240501 19:56:04.359169 21320 ts_itest-base.cc:113] --------------
I20240501 19:56:04.359179 21320 ts_itest-base.cc:114] 5 tablet servers
I20240501 19:56:04.359189 21320 ts_itest-base.cc:115] 3 replicas per TS
I20240501 19:56:04.359196 21320 ts_itest-base.cc:116] --------------
2024-05-01T19:56:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T19:56:04Z Disabled control of system clock
I20240501 19:56:04.365785 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:35137
--webserver_interface=127.20.210.62
--webserver_port=0
--builtin_ntp_servers=127.20.210.20:43351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.210.62:35137
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240501 19:56:04.458153 22167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:04.458410 22167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:04.458472 22167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:04.461036 22167 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:04.461099 22167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 19:56:04.461130 22167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:04.461166 22167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 19:56:04.461205 22167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 19:56:04.464534 22167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:43351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.20.210.62:35137
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:35137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.20.210.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.22167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:04.465247 22167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:04.466166 22167 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:04.472915 22172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:04.473603 22173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:04.474360 22175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:04.475040 22167 server_base.cc:1034] running on GCE node
I20240501 19:56:04.475435 22167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:04.476109 22167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:04.477241 22167 hybrid_clock.cc:648] HybridClock initialized: now 1714593364477220 us; error 34 us; skew 500 ppm
I20240501 19:56:04.478788 22167 webserver.cc:460] Webserver started at http://127.20.210.62:46497/ using document root <none> and password file <none>
I20240501 19:56:04.479220 22167 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:04.479270 22167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:04.479444 22167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:04.480813 22167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "77dc002b325e41c4a7a304355c14bd38"
format_stamp: "Formatted at 2024-05-01 19:56:04 on dist-test-slave-vgh4"
I20240501 19:56:04.481247 22167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "77dc002b325e41c4a7a304355c14bd38"
format_stamp: "Formatted at 2024-05-01 19:56:04 on dist-test-slave-vgh4"
I20240501 19:56:04.484237 22167 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240501 19:56:04.486068 22181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:04.486922 22167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:04.487033 22167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
uuid: "77dc002b325e41c4a7a304355c14bd38"
format_stamp: "Formatted at 2024-05-01 19:56:04 on dist-test-slave-vgh4"
I20240501 19:56:04.487104 22167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:04.504573 22167 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:04.505080 22167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:04.505213 22167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:04.510592 22167 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.62:35137
I20240501 19:56:04.510603 22233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.62:35137 every 8 connection(s)
I20240501 19:56:04.511449 22167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 19:56:04.513691 22234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:04.518141 22234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38: Bootstrap starting.
I20240501 19:56:04.520015 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 22167
I20240501 19:56:04.520054 22234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:04.520112 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 19:56:04.520653 22234 log.cc:824] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:04.522428 22234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38: No bootstrap required, opened a new log
I20240501 19:56:04.525305 22234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77dc002b325e41c4a7a304355c14bd38" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 35137 } }
I20240501 19:56:04.525472 22234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:04.525519 22234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77dc002b325e41c4a7a304355c14bd38, State: Initialized, Role: FOLLOWER
I20240501 19:56:04.525916 22234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77dc002b325e41c4a7a304355c14bd38" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 35137 } }
I20240501 19:56:04.526027 22234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:56:04.526084 22234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:56:04.526191 22234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:04.526834 22234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77dc002b325e41c4a7a304355c14bd38" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 35137 } }
I20240501 19:56:04.527235 22234 leader_election.cc:302] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77dc002b325e41c4a7a304355c14bd38; no voters:
I20240501 19:56:04.527479 22234 leader_election.cc:288] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:56:04.527551 22239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:04.527724 22239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [term 1 LEADER]: Becoming Leader. State: Replica: 77dc002b325e41c4a7a304355c14bd38, State: Running, Role: LEADER
I20240501 19:56:04.527973 22239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77dc002b325e41c4a7a304355c14bd38" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 35137 } }
I20240501 19:56:04.528924 22239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77dc002b325e41c4a7a304355c14bd38. Latest consensus state: current_term: 1 leader_uuid: "77dc002b325e41c4a7a304355c14bd38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77dc002b325e41c4a7a304355c14bd38" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 35137 } } }
I20240501 19:56:04.529028 22239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [sys.catalog]: This master's current role is: LEADER
I20240501 19:56:04.529318 22234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 19:56:04.529347 22240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77dc002b325e41c4a7a304355c14bd38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77dc002b325e41c4a7a304355c14bd38" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 35137 } } }
I20240501 19:56:04.529431 22243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 19:56:04.529443 22240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38 [sys.catalog]: This master's current role is: LEADER
I20240501 19:56:04.531955 22243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 19:56:04.536067 22243 catalog_manager.cc:1347] Generated new cluster ID: 1c100bc5f4424837953d1b13d07d106c
I20240501 19:56:04.536126 22243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 19:56:04.546386 22243 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 19:56:04.547098 22243 catalog_manager.cc:1504] Loading token signing keys...
I20240501 19:56:04.565632 22243 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 77dc002b325e41c4a7a304355c14bd38: Generated new TSK 0
I20240501 19:56:04.566210 22243 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 19:56:04.574512 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.1:0
--local_ip_for_outbound_sockets=127.20.210.1
--webserver_interface=127.20.210.1
--webserver_port=0
--tserver_master_addrs=127.20.210.62:35137
--builtin_ntp_servers=127.20.210.20:43351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 19:56:04.669811 22258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:04.669996 22258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:04.670059 22258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:04.672842 22258 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:04.672901 22258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:04.672979 22258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.1
I20240501 19:56:04.676095 22258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:43351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.20.210.1
--webserver_port=0
--tserver_master_addrs=127.20.210.62:35137
--never_fsync=true
--heap_profile_path=/tmp/kudu.22258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.1
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:04.676834 22258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:04.677819 22258 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:04.685523 22264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:04.686141 22263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:04.686769 22266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:04.687424 22258 server_base.cc:1034] running on GCE node
I20240501 19:56:04.687757 22258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:04.688194 22258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:04.689356 22258 hybrid_clock.cc:648] HybridClock initialized: now 1714593364689334 us; error 38 us; skew 500 ppm
I20240501 19:56:04.691012 22258 webserver.cc:460] Webserver started at http://127.20.210.1:37161/ using document root <none> and password file <none>
I20240501 19:56:04.691494 22258 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:04.691553 22258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:04.691711 22258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:04.693115 22258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "1c1b68012c5f4c219540f32e5296a0c3"
format_stamp: "Formatted at 2024-05-01 19:56:04 on dist-test-slave-vgh4"
I20240501 19:56:04.693521 22258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "1c1b68012c5f4c219540f32e5296a0c3"
format_stamp: "Formatted at 2024-05-01 19:56:04 on dist-test-slave-vgh4"
I20240501 19:56:04.696359 22258 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20240501 19:56:04.698312 22272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:04.699151 22258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240501 19:56:04.699246 22258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "1c1b68012c5f4c219540f32e5296a0c3"
format_stamp: "Formatted at 2024-05-01 19:56:04 on dist-test-slave-vgh4"
I20240501 19:56:04.699318 22258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:04.721817 22258 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:04.722318 22258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:04.722462 22258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:04.722899 22258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:04.723619 22258 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:04.723664 22258 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:04.723714 22258 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:04.723745 22258 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:04.731843 22258 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.1:41293
I20240501 19:56:04.731899 22385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.1:41293 every 8 connection(s)
I20240501 19:56:04.732694 22258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 19:56:04.739329 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 22258
I20240501 19:56:04.739415 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 19:56:04.740867 22386 heartbeater.cc:347] Connected to a master server at 127.20.210.62:35137
I20240501 19:56:04.741051 22386 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:04.741551 22386 heartbeater.cc:511] Master 127.20.210.62:35137 requested a full tablet report, sending...
I20240501 19:56:04.741724 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.2:0
--local_ip_for_outbound_sockets=127.20.210.2
--webserver_interface=127.20.210.2
--webserver_port=0
--tserver_master_addrs=127.20.210.62:35137
--builtin_ntp_servers=127.20.210.20:43351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:56:04.742993 22197 ts_manager.cc:194] Registered new tserver with Master: 1c1b68012c5f4c219540f32e5296a0c3 (127.20.210.1:41293)
I20240501 19:56:04.744103 22197 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.1:36995
W20240501 19:56:04.836020 22389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:04.836190 22389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:04.836257 22389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:04.838860 22389 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:04.838917 22389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:04.839002 22389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.2
I20240501 19:56:04.842146 22389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:43351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.20.210.2
--webserver_port=0
--tserver_master_addrs=127.20.210.62:35137
--never_fsync=true
--heap_profile_path=/tmp/kudu.22389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.2
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:04.842854 22389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:04.843725 22389 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:04.851193 22394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:04.851895 22397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:04.852659 22395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:04.853320 22389 server_base.cc:1034] running on GCE node
I20240501 19:56:04.853612 22389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:04.854066 22389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:04.855208 22389 hybrid_clock.cc:648] HybridClock initialized: now 1714593364855199 us; error 25 us; skew 500 ppm
I20240501 19:56:04.856808 22389 webserver.cc:460] Webserver started at http://127.20.210.2:44979/ using document root <none> and password file <none>
I20240501 19:56:04.857266 22389 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:04.857322 22389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:04.857489 22389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:04.859011 22389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "c062e5f8a9714907b1de37f2233b8b14"
format_stamp: "Formatted at 2024-05-01 19:56:04 on dist-test-slave-vgh4"
I20240501 19:56:04.859429 22389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "c062e5f8a9714907b1de37f2233b8b14"
format_stamp: "Formatted at 2024-05-01 19:56:04 on dist-test-slave-vgh4"
I20240501 19:56:04.862218 22389 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 19:56:04.864092 22403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:04.864924 22389 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240501 19:56:04.865028 22389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "c062e5f8a9714907b1de37f2233b8b14"
format_stamp: "Formatted at 2024-05-01 19:56:04 on dist-test-slave-vgh4"
I20240501 19:56:04.865097 22389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:04.880754 22389 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:04.881201 22389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:04.881378 22389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:04.881814 22389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:04.882727 22389 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:04.882776 22389 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:04.882823 22389 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:04.882850 22389 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:04.889942 22389 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.2:45393
I20240501 19:56:04.889994 22516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.2:45393 every 8 connection(s)
I20240501 19:56:04.890769 22389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 19:56:04.896661 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 22389
I20240501 19:56:04.896767 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 19:56:04.898536 22517 heartbeater.cc:347] Connected to a master server at 127.20.210.62:35137
I20240501 19:56:04.898730 22517 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:04.898756 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.3:0
--local_ip_for_outbound_sockets=127.20.210.3
--webserver_interface=127.20.210.3
--webserver_port=0
--tserver_master_addrs=127.20.210.62:35137
--builtin_ntp_servers=127.20.210.20:43351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:56:04.899246 22517 heartbeater.cc:511] Master 127.20.210.62:35137 requested a full tablet report, sending...
I20240501 19:56:04.900143 22198 ts_manager.cc:194] Registered new tserver with Master: c062e5f8a9714907b1de37f2233b8b14 (127.20.210.2:45393)
I20240501 19:56:04.900622 22198 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.2:55931
W20240501 19:56:04.991648 22520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:04.991840 22520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:04.991911 22520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:04.994725 22520 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:04.994791 22520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:04.994895 22520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.3
I20240501 19:56:04.998023 22520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:43351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.20.210.3
--webserver_port=0
--tserver_master_addrs=127.20.210.62:35137
--never_fsync=true
--heap_profile_path=/tmp/kudu.22520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.3
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:04.998772 22520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:04.999715 22520 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:05.007556 22526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:05.008205 22525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:05.008795 22528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:05.009454 22520 server_base.cc:1034] running on GCE node
I20240501 19:56:05.009730 22520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:05.010154 22520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:05.011309 22520 hybrid_clock.cc:648] HybridClock initialized: now 1714593365011303 us; error 25 us; skew 500 ppm
I20240501 19:56:05.012909 22520 webserver.cc:460] Webserver started at http://127.20.210.3:40913/ using document root <none> and password file <none>
I20240501 19:56:05.013360 22520 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:05.013414 22520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:05.013559 22520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:05.014961 22520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "b36f29590319494aa57dde3f06bc9401"
format_stamp: "Formatted at 2024-05-01 19:56:05 on dist-test-slave-vgh4"
I20240501 19:56:05.015375 22520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "b36f29590319494aa57dde3f06bc9401"
format_stamp: "Formatted at 2024-05-01 19:56:05 on dist-test-slave-vgh4"
I20240501 19:56:05.018252 22520 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 19:56:05.020124 22534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:05.020974 22520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:05.021068 22520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "b36f29590319494aa57dde3f06bc9401"
format_stamp: "Formatted at 2024-05-01 19:56:05 on dist-test-slave-vgh4"
I20240501 19:56:05.021142 22520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:05.038077 22520 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:05.038559 22520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:05.038694 22520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:05.039109 22520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:05.039851 22520 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:05.039896 22520 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:05.039948 22520 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:05.039973 22520 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:05.047223 22520 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.3:42963
I20240501 19:56:05.047266 22647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.3:42963 every 8 connection(s)
I20240501 19:56:05.047982 22520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 19:56:05.054533 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 22520
I20240501 19:56:05.054625 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 19:56:05.055193 22648 heartbeater.cc:347] Connected to a master server at 127.20.210.62:35137
I20240501 19:56:05.055379 22648 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:05.055812 22648 heartbeater.cc:511] Master 127.20.210.62:35137 requested a full tablet report, sending...
I20240501 19:56:05.056491 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.4:0
--local_ip_for_outbound_sockets=127.20.210.4
--webserver_interface=127.20.210.4
--webserver_port=0
--tserver_master_addrs=127.20.210.62:35137
--builtin_ntp_servers=127.20.210.20:43351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:56:05.056689 22198 ts_manager.cc:194] Registered new tserver with Master: b36f29590319494aa57dde3f06bc9401 (127.20.210.3:42963)
I20240501 19:56:05.057339 22198 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.3:34137
W20240501 19:56:05.153169 22651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:05.153425 22651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:05.153496 22651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:05.157512 22651 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:05.157579 22651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:05.157671 22651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.4
I20240501 19:56:05.161868 22651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:43351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.4:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.20.210.4
--webserver_port=0
--tserver_master_addrs=127.20.210.62:35137
--never_fsync=true
--heap_profile_path=/tmp/kudu.22651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.4
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:05.162871 22651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:05.163911 22651 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:05.172062 22657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:05.172901 22656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:05.173462 22659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:05.174264 22651 server_base.cc:1034] running on GCE node
I20240501 19:56:05.174589 22651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:05.175096 22651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:05.176249 22651 hybrid_clock.cc:648] HybridClock initialized: now 1714593365176237 us; error 33 us; skew 500 ppm
I20240501 19:56:05.177938 22651 webserver.cc:460] Webserver started at http://127.20.210.4:44425/ using document root <none> and password file <none>
I20240501 19:56:05.178465 22651 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:05.178524 22651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:05.178704 22651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:05.180151 22651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "40f562492fb34b25934cfa6142c728a3"
format_stamp: "Formatted at 2024-05-01 19:56:05 on dist-test-slave-vgh4"
I20240501 19:56:05.180599 22651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "40f562492fb34b25934cfa6142c728a3"
format_stamp: "Formatted at 2024-05-01 19:56:05 on dist-test-slave-vgh4"
I20240501 19:56:05.183594 22651 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 19:56:05.185549 22665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:05.186429 22651 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 19:56:05.186522 22651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "40f562492fb34b25934cfa6142c728a3"
format_stamp: "Formatted at 2024-05-01 19:56:05 on dist-test-slave-vgh4"
I20240501 19:56:05.186592 22651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:05.195698 22651 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:05.196131 22651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:05.196264 22651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:05.196676 22651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:05.197410 22651 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:05.197464 22651 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:05.197530 22651 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:05.197561 22651 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:05.205224 22651 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.4:39471
I20240501 19:56:05.205276 22778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.4:39471 every 8 connection(s)
I20240501 19:56:05.205958 22651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 19:56:05.211545 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 22651
I20240501 19:56:05.211655 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 19:56:05.213322 22779 heartbeater.cc:347] Connected to a master server at 127.20.210.62:35137
I20240501 19:56:05.213492 22779 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:05.213703 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.5:0
--local_ip_for_outbound_sockets=127.20.210.5
--webserver_interface=127.20.210.5
--webserver_port=0
--tserver_master_addrs=127.20.210.62:35137
--builtin_ntp_servers=127.20.210.20:43351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:56:05.213963 22779 heartbeater.cc:511] Master 127.20.210.62:35137 requested a full tablet report, sending...
I20240501 19:56:05.214862 22198 ts_manager.cc:194] Registered new tserver with Master: 40f562492fb34b25934cfa6142c728a3 (127.20.210.4:39471)
I20240501 19:56:05.215593 22198 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.4:56279
W20240501 19:56:05.310175 22782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:05.310351 22782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:05.310393 22782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:05.313292 22782 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:05.313342 22782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:05.313400 22782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.5
I20240501 19:56:05.316346 22782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:43351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.5:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.20.210.5
--webserver_port=0
--tserver_master_addrs=127.20.210.62:35137
--never_fsync=true
--heap_profile_path=/tmp/kudu.22782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.5
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:05.317003 22782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:05.317888 22782 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:05.325495 22788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:05.326159 22790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:05.326856 22787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:05.327626 22782 server_base.cc:1034] running on GCE node
I20240501 19:56:05.327984 22782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:05.328471 22782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:05.329619 22782 hybrid_clock.cc:648] HybridClock initialized: now 1714593365329607 us; error 25 us; skew 500 ppm
I20240501 19:56:05.331321 22782 webserver.cc:460] Webserver started at http://127.20.210.5:37001/ using document root <none> and password file <none>
I20240501 19:56:05.331821 22782 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:05.331872 22782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:05.332020 22782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:05.333441 22782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "f0754d1467f14f1eae80ce7122728c6a"
format_stamp: "Formatted at 2024-05-01 19:56:05 on dist-test-slave-vgh4"
I20240501 19:56:05.333824 22782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "f0754d1467f14f1eae80ce7122728c6a"
format_stamp: "Formatted at 2024-05-01 19:56:05 on dist-test-slave-vgh4"
I20240501 19:56:05.336974 22782 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20240501 19:56:05.338879 22796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:05.339715 22782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240501 19:56:05.339808 22782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "f0754d1467f14f1eae80ce7122728c6a"
format_stamp: "Formatted at 2024-05-01 19:56:05 on dist-test-slave-vgh4"
I20240501 19:56:05.339880 22782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:05.353538 22782 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:05.353981 22782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:05.354116 22782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:05.354557 22782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:05.355260 22782 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:05.355311 22782 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:05.355369 22782 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:05.355394 22782 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:05.363072 22782 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.5:35033
I20240501 19:56:05.363094 22909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.5:35033 every 8 connection(s)
I20240501 19:56:05.363827 22782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 19:56:05.368562 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 22782
I20240501 19:56:05.368685 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240501 19:56:05.371357 22910 heartbeater.cc:347] Connected to a master server at 127.20.210.62:35137
I20240501 19:56:05.371538 22910 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:05.371958 22910 heartbeater.cc:511] Master 127.20.210.62:35137 requested a full tablet report, sending...
I20240501 19:56:05.372814 22198 ts_manager.cc:194] Registered new tserver with Master: f0754d1467f14f1eae80ce7122728c6a (127.20.210.5:35033)
I20240501 19:56:05.373308 22198 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.5:50449
I20240501 19:56:05.382226 21320 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240501 19:56:05.389519 22198 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36642:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240501 19:56:05.405184 22451 tablet_service.cc:1467] Processing CreateTablet for tablet 6224227d874e4c05a88d507aaaaafb66 (DEFAULT_TABLE table=TestTable [id=c11323e34bea4d4097fd4305c2592fb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:05.405329 22582 tablet_service.cc:1467] Processing CreateTablet for tablet 6224227d874e4c05a88d507aaaaafb66 (DEFAULT_TABLE table=TestTable [id=c11323e34bea4d4097fd4305c2592fb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:05.405329 22320 tablet_service.cc:1467] Processing CreateTablet for tablet 6224227d874e4c05a88d507aaaaafb66 (DEFAULT_TABLE table=TestTable [id=c11323e34bea4d4097fd4305c2592fb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:05.405934 22451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6224227d874e4c05a88d507aaaaafb66. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:05.406067 22320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6224227d874e4c05a88d507aaaaafb66. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:05.406067 22582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6224227d874e4c05a88d507aaaaafb66. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:05.409934 22927 tablet_bootstrap.cc:492] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14: Bootstrap starting.
I20240501 19:56:05.411731 22929 tablet_bootstrap.cc:492] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401: Bootstrap starting.
I20240501 19:56:05.411893 22927 tablet_bootstrap.cc:654] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:05.412510 22927 log.cc:824] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:05.413223 22929 tablet_bootstrap.cc:654] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:05.413766 22929 log.cc:824] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:05.413786 22927 tablet_bootstrap.cc:492] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14: No bootstrap required, opened a new log
I20240501 19:56:05.413918 22927 ts_tablet_manager.cc:1367] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 19:56:05.414314 22928 tablet_bootstrap.cc:492] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3: Bootstrap starting.
I20240501 19:56:05.415267 22929 tablet_bootstrap.cc:492] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401: No bootstrap required, opened a new log
I20240501 19:56:05.415419 22929 ts_tablet_manager.cc:1367] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 19:56:05.416393 22928 tablet_bootstrap.cc:654] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:05.416735 22927 raft_consensus.cc:348] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c062e5f8a9714907b1de37f2233b8b14" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 45393 } } peers { permanent_uuid: "b36f29590319494aa57dde3f06bc9401" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 42963 } } peers { permanent_uuid: "1c1b68012c5f4c219540f32e5296a0c3" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 41293 } }
I20240501 19:56:05.416898 22927 raft_consensus.cc:374] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:05.416949 22927 raft_consensus.cc:729] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c062e5f8a9714907b1de37f2233b8b14, State: Initialized, Role: FOLLOWER
I20240501 19:56:05.417049 22928 log.cc:824] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:05.417297 22927 consensus_queue.cc:260] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c062e5f8a9714907b1de37f2233b8b14" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 45393 } } peers { permanent_uuid: "b36f29590319494aa57dde3f06bc9401" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 42963 } } peers { permanent_uuid: "1c1b68012c5f4c219540f32e5296a0c3" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 41293 } }
I20240501 19:56:05.417815 22517 heartbeater.cc:503] Master 127.20.210.62:35137 was elected leader, sending a full tablet report...
I20240501 19:56:05.418507 22928 tablet_bootstrap.cc:492] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3: No bootstrap required, opened a new log
I20240501 19:56:05.418645 22928 ts_tablet_manager.cc:1367] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240501 19:56:05.418808 22927 ts_tablet_manager.cc:1398] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240501 19:56:05.418721 22929 raft_consensus.cc:348] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c062e5f8a9714907b1de37f2233b8b14" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 45393 } } peers { permanent_uuid: "b36f29590319494aa57dde3f06bc9401" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 42963 } } peers { permanent_uuid: "1c1b68012c5f4c219540f32e5296a0c3" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 41293 } }
I20240501 19:56:05.418912 22929 raft_consensus.cc:374] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:05.418963 22929 raft_consensus.cc:729] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36f29590319494aa57dde3f06bc9401, State: Initialized, Role: FOLLOWER
I20240501 19:56:05.419312 22929 consensus_queue.cc:260] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c062e5f8a9714907b1de37f2233b8b14" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 45393 } } peers { permanent_uuid: "b36f29590319494aa57dde3f06bc9401" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 42963 } } peers { permanent_uuid: "1c1b68012c5f4c219540f32e5296a0c3" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 41293 } }
I20240501 19:56:05.420038 22648 heartbeater.cc:503] Master 127.20.210.62:35137 was elected leader, sending a full tablet report...
I20240501 19:56:05.420064 22929 ts_tablet_manager.cc:1398] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20240501 19:56:05.421633 22928 raft_consensus.cc:348] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c062e5f8a9714907b1de37f2233b8b14" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 45393 } } peers { permanent_uuid: "b36f29590319494aa57dde3f06bc9401" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 42963 } } peers { permanent_uuid: "1c1b68012c5f4c219540f32e5296a0c3" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 41293 } }
I20240501 19:56:05.421838 22928 raft_consensus.cc:374] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:05.421896 22928 raft_consensus.cc:729] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1b68012c5f4c219540f32e5296a0c3, State: Initialized, Role: FOLLOWER
I20240501 19:56:05.422283 22928 consensus_queue.cc:260] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c062e5f8a9714907b1de37f2233b8b14" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 45393 } } peers { permanent_uuid: "b36f29590319494aa57dde3f06bc9401" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 42963 } } peers { permanent_uuid: "1c1b68012c5f4c219540f32e5296a0c3" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 41293 } }
I20240501 19:56:05.422870 22928 ts_tablet_manager.cc:1398] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240501 19:56:05.422909 22386 heartbeater.cc:503] Master 127.20.210.62:35137 was elected leader, sending a full tablet report...
W20240501 19:56:05.484009 22387 tablet.cc:2369] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 19:56:05.548926 22649 tablet.cc:2369] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:56:05.568778 22935 raft_consensus.cc:482] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:05.568928 22935 raft_consensus.cc:504] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c062e5f8a9714907b1de37f2233b8b14" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 45393 } } peers { permanent_uuid: "b36f29590319494aa57dde3f06bc9401" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 42963 } } peers { permanent_uuid: "1c1b68012c5f4c219540f32e5296a0c3" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 41293 } }
I20240501 19:56:05.569823 22935 leader_election.cc:288] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c062e5f8a9714907b1de37f2233b8b14 (127.20.210.2:45393), b36f29590319494aa57dde3f06bc9401 (127.20.210.3:42963)
I20240501 19:56:05.573218 22602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6224227d874e4c05a88d507aaaaafb66" candidate_uuid: "1c1b68012c5f4c219540f32e5296a0c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36f29590319494aa57dde3f06bc9401" is_pre_election: true
I20240501 19:56:05.573239 22471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6224227d874e4c05a88d507aaaaafb66" candidate_uuid: "1c1b68012c5f4c219540f32e5296a0c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c062e5f8a9714907b1de37f2233b8b14" is_pre_election: true
I20240501 19:56:05.573436 22602 raft_consensus.cc:2454] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c1b68012c5f4c219540f32e5296a0c3 in term 0.
I20240501 19:56:05.573450 22471 raft_consensus.cc:2454] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c1b68012c5f4c219540f32e5296a0c3 in term 0.
I20240501 19:56:05.573798 22274 leader_election.cc:302] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c1b68012c5f4c219540f32e5296a0c3, b36f29590319494aa57dde3f06bc9401; no voters:
I20240501 19:56:05.573997 22935 raft_consensus.cc:2781] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:05.574059 22935 raft_consensus.cc:482] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:05.574083 22935 raft_consensus.cc:3037] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:05.574864 22935 raft_consensus.cc:504] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c062e5f8a9714907b1de37f2233b8b14" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 45393 } } peers { permanent_uuid: "b36f29590319494aa57dde3f06bc9401" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 42963 } } peers { permanent_uuid: "1c1b68012c5f4c219540f32e5296a0c3" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 41293 } }
I20240501 19:56:05.575152 22935 leader_election.cc:288] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [CANDIDATE]: Term 1 election: Requested vote from peers c062e5f8a9714907b1de37f2233b8b14 (127.20.210.2:45393), b36f29590319494aa57dde3f06bc9401 (127.20.210.3:42963)
I20240501 19:56:05.575426 22602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6224227d874e4c05a88d507aaaaafb66" candidate_uuid: "1c1b68012c5f4c219540f32e5296a0c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36f29590319494aa57dde3f06bc9401"
I20240501 19:56:05.575431 22471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6224227d874e4c05a88d507aaaaafb66" candidate_uuid: "1c1b68012c5f4c219540f32e5296a0c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c062e5f8a9714907b1de37f2233b8b14"
I20240501 19:56:05.575515 22602 raft_consensus.cc:3037] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:05.575523 22471 raft_consensus.cc:3037] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:05.576215 22602 raft_consensus.cc:2454] T 6224227d874e4c05a88d507aaaaafb66 P b36f29590319494aa57dde3f06bc9401 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1b68012c5f4c219540f32e5296a0c3 in term 1.
I20240501 19:56:05.576290 22471 raft_consensus.cc:2454] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c1b68012c5f4c219540f32e5296a0c3 in term 1.
I20240501 19:56:05.576442 22274 leader_election.cc:302] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c1b68012c5f4c219540f32e5296a0c3, b36f29590319494aa57dde3f06bc9401; no voters:
I20240501 19:56:05.576598 22935 raft_consensus.cc:2781] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:05.576759 22935 raft_consensus.cc:686] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [term 1 LEADER]: Becoming Leader. State: Replica: 1c1b68012c5f4c219540f32e5296a0c3, State: Running, Role: LEADER
I20240501 19:56:05.577018 22935 consensus_queue.cc:237] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c062e5f8a9714907b1de37f2233b8b14" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 45393 } } peers { permanent_uuid: "b36f29590319494aa57dde3f06bc9401" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 42963 } } peers { permanent_uuid: "1c1b68012c5f4c219540f32e5296a0c3" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 41293 } }
I20240501 19:56:05.579519 22196 catalog_manager.cc:5484] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c1b68012c5f4c219540f32e5296a0c3 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "1c1b68012c5f4c219540f32e5296a0c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c062e5f8a9714907b1de37f2233b8b14" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b36f29590319494aa57dde3f06bc9401" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 42963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c1b68012c5f4c219540f32e5296a0c3" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 41293 } health_report { overall_health: HEALTHY } } }
I20240501 19:56:05.592238 21320 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240501 19:56:05.593649 21320 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 1c1b68012c5f4c219540f32e5296a0c3 to finish bootstrapping
I20240501 19:56:05.597940 21320 ts_itest-base.cc:233] Waiting for 1 tablets on tserver c062e5f8a9714907b1de37f2233b8b14 to finish bootstrapping
I20240501 19:56:05.601728 21320 ts_itest-base.cc:233] Waiting for 1 tablets on tserver b36f29590319494aa57dde3f06bc9401 to finish bootstrapping
I20240501 19:56:05.605775 21320 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 40f562492fb34b25934cfa6142c728a3 to finish bootstrapping
I20240501 19:56:05.610755 21320 ts_itest-base.cc:233] Waiting for 0 tablets on tserver f0754d1467f14f1eae80ce7122728c6a to finish bootstrapping
W20240501 19:56:05.641870 22518 tablet.cc:2369] T 6224227d874e4c05a88d507aaaaafb66 P c062e5f8a9714907b1de37f2233b8b14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:56:05.774784 22320 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:05.774823 22451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:05.776149 22582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:05.776646 22844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:05.783857 22713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:05.856492 22713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:05.858631 22320 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:05.859253 22451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:05.860009 22844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:05.877848 22582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:05.952536 22320 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:05.953366 22451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:05.953692 22844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:05.955183 22713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:05.955636 22582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:06.031826 22935 consensus_queue.cc:1035] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b36f29590319494aa57dde3f06bc9401" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 42963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:06.036443 22935 consensus_queue.cc:1035] T 6224227d874e4c05a88d507aaaaafb66 P 1c1b68012c5f4c219540f32e5296a0c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c062e5f8a9714907b1de37f2233b8b14" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:06.152822 22713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:06.153131 22451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:06.154028 22582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:06.154083 22320 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:06.154299 22844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:06.217147 22779 heartbeater.cc:503] Master 127.20.210.62:35137 was elected leader, sending a full tablet report...
I20240501 19:56:06.238133 22713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:06.238579 22451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:06.239722 22320 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:06.247310 22582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:06.248551 22844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:06.339928 22320 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:06.340816 22713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:06.342865 22844 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:06.346694 22451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:06.350790 22582 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:06.374962 22910 heartbeater.cc:503] Master 127.20.210.62:35137 was elected leader, sending a full tablet report...
I20240501 19:56:06.381320 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 22258
I20240501 19:56:06.386947 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 22389
I20240501 19:56:06.391989 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 22520
I20240501 19:56:06.396651 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 22651
I20240501 19:56:06.401240 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 22782
I20240501 19:56:06.406105 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 22167
2024-05-01T19:56:06Z chronyd exiting
[ OK ] RebalanceStartSafetyTest.TooManyIgnoredTservers/1 (2068 ms)
[----------] 1 test from RebalanceStartSafetyTest (2068 ms total)
[----------] 1 test from RebalanceParamTest
[ RUN ] RebalanceParamTest.Rebalance/7
I20240501 19:56:06.427172 21320 test_util.cc:268] Using random seed: -527808990
I20240501 19:56:06.427765 21320 ts_itest-base.cc:112] Starting cluster with:
I20240501 19:56:06.427798 21320 ts_itest-base.cc:113] --------------
I20240501 19:56:06.427814 21320 ts_itest-base.cc:114] 7 tablet servers
I20240501 19:56:06.427826 21320 ts_itest-base.cc:115] 5 replicas per TS
I20240501 19:56:06.427858 21320 ts_itest-base.cc:116] --------------
2024-05-01T19:56:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T19:56:06Z Disabled control of system clock
I20240501 19:56:06.434111 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:37935
--webserver_interface=127.20.210.62
--webserver_port=0
--builtin_ntp_servers=127.20.210.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.210.62:37935
--allow_unsafe_replication_factor
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--tserver_unresponsive_timeout_ms=3000 with env {}
W20240501 19:56:06.523293 23153 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240501 19:56:06.523468 23153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:06.523507 23153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:06.523564 23153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:06.526168 23153 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:06.526230 23153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 19:56:06.526280 23153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:06.526316 23153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 19:56:06.526350 23153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 19:56:06.529574 23153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.20.210.62:37935
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:37935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.20.210.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.23153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:06.530285 23153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:06.531155 23153 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:06.537540 23158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:06.538331 23161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:06.539011 23159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:06.539700 23153 server_base.cc:1034] running on GCE node
I20240501 19:56:06.540088 23153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:06.540760 23153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:06.541899 23153 hybrid_clock.cc:648] HybridClock initialized: now 1714593366541888 us; error 22 us; skew 500 ppm
I20240501 19:56:06.543380 23153 webserver.cc:460] Webserver started at http://127.20.210.62:39691/ using document root <none> and password file <none>
I20240501 19:56:06.543848 23153 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:06.543926 23153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:06.544080 23153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:06.545460 23153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "bb28524133a341a69c1f36bc25de2d8d"
format_stamp: "Formatted at 2024-05-01 19:56:06 on dist-test-slave-vgh4"
I20240501 19:56:06.545856 23153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "bb28524133a341a69c1f36bc25de2d8d"
format_stamp: "Formatted at 2024-05-01 19:56:06 on dist-test-slave-vgh4"
I20240501 19:56:06.548545 23153 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.002s
I20240501 19:56:06.550294 23167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:06.551077 23153 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:06.551191 23153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
uuid: "bb28524133a341a69c1f36bc25de2d8d"
format_stamp: "Formatted at 2024-05-01 19:56:06 on dist-test-slave-vgh4"
I20240501 19:56:06.551265 23153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:06.575563 23153 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:06.576007 23153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:06.576129 23153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:06.581374 23153 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.62:37935
I20240501 19:56:06.581384 23219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.62:37935 every 8 connection(s)
I20240501 19:56:06.582167 23153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 19:56:06.584215 23220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:06.588179 23220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d: Bootstrap starting.
I20240501 19:56:06.588587 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 23153
I20240501 19:56:06.588698 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 19:56:06.590417 23220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:06.590972 23220 log.cc:824] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:06.592271 23220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d: No bootstrap required, opened a new log
I20240501 19:56:06.594417 23220 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb28524133a341a69c1f36bc25de2d8d" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 37935 } }
I20240501 19:56:06.594563 23220 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:06.594622 23220 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb28524133a341a69c1f36bc25de2d8d, State: Initialized, Role: FOLLOWER
I20240501 19:56:06.595018 23220 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb28524133a341a69c1f36bc25de2d8d" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 37935 } }
I20240501 19:56:06.595150 23220 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:56:06.595202 23220 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:56:06.595292 23220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:06.595870 23220 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb28524133a341a69c1f36bc25de2d8d" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 37935 } }
I20240501 19:56:06.596155 23220 leader_election.cc:302] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb28524133a341a69c1f36bc25de2d8d; no voters:
I20240501 19:56:06.596388 23220 leader_election.cc:288] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:56:06.596710 23225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:06.596877 23225 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [term 1 LEADER]: Becoming Leader. State: Replica: bb28524133a341a69c1f36bc25de2d8d, State: Running, Role: LEADER
I20240501 19:56:06.597126 23225 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb28524133a341a69c1f36bc25de2d8d" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 37935 } }
I20240501 19:56:06.597205 23220 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [sys.catalog]: configured and running, proceeding with master startup.
I20240501 19:56:06.598381 23227 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb28524133a341a69c1f36bc25de2d8d. Latest consensus state: current_term: 1 leader_uuid: "bb28524133a341a69c1f36bc25de2d8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb28524133a341a69c1f36bc25de2d8d" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 37935 } } }
I20240501 19:56:06.598402 23226 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb28524133a341a69c1f36bc25de2d8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb28524133a341a69c1f36bc25de2d8d" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 37935 } } }
I20240501 19:56:06.598542 23226 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [sys.catalog]: This master's current role is: LEADER
I20240501 19:56:06.598560 23227 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d [sys.catalog]: This master's current role is: LEADER
W20240501 19:56:06.601079 23240 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240501 19:56:06.601202 23240 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240501 19:56:06.601336 23237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 19:56:06.602015 23237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 19:56:06.606356 23237 catalog_manager.cc:1347] Generated new cluster ID: 8bf980e99b164351ac1f22992c653f6a
I20240501 19:56:06.606405 23237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 19:56:06.611936 23237 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 19:56:06.612677 23237 catalog_manager.cc:1504] Loading token signing keys...
I20240501 19:56:06.627689 23237 catalog_manager.cc:5857] T 00000000000000000000000000000000 P bb28524133a341a69c1f36bc25de2d8d: Generated new TSK 0
I20240501 19:56:06.628177 23237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 19:56:06.631898 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.1:0
--local_ip_for_outbound_sockets=127.20.210.1
--webserver_interface=127.20.210.1
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--builtin_ntp_servers=127.20.210.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 19:56:06.724474 23244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:06.724645 23244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:06.724715 23244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:06.727308 23244 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:06.727378 23244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:06.727573 23244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.1
I20240501 19:56:06.730584 23244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.20.210.1
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--never_fsync=true
--heap_profile_path=/tmp/kudu.23244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.1
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:06.731201 23244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:06.732121 23244 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:06.739701 23249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:06.740309 23252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:06.740944 23250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:06.741648 23244 server_base.cc:1034] running on GCE node
I20240501 19:56:06.741959 23244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:06.742465 23244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:06.743618 23244 hybrid_clock.cc:648] HybridClock initialized: now 1714593366743604 us; error 27 us; skew 500 ppm
I20240501 19:56:06.745170 23244 webserver.cc:460] Webserver started at http://127.20.210.1:37241/ using document root <none> and password file <none>
I20240501 19:56:06.745617 23244 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:06.745666 23244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:06.745841 23244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:06.747357 23244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "ad9452840e4e4184a640fb63dab754fd"
format_stamp: "Formatted at 2024-05-01 19:56:06 on dist-test-slave-vgh4"
I20240501 19:56:06.747828 23244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "ad9452840e4e4184a640fb63dab754fd"
format_stamp: "Formatted at 2024-05-01 19:56:06 on dist-test-slave-vgh4"
I20240501 19:56:06.750490 23244 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20240501 19:56:06.752295 23258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:06.753144 23244 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:06.753249 23244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "ad9452840e4e4184a640fb63dab754fd"
format_stamp: "Formatted at 2024-05-01 19:56:06 on dist-test-slave-vgh4"
I20240501 19:56:06.753325 23244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:06.774052 23244 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:06.774552 23244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:06.774686 23244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:06.775105 23244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:06.775815 23244 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:06.775877 23244 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:06.775926 23244 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:06.775956 23244 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:06.783553 23244 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.1:35481
I20240501 19:56:06.783576 23371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.1:35481 every 8 connection(s)
I20240501 19:56:06.784303 23244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 19:56:06.786747 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 23244
I20240501 19:56:06.786859 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 19:56:06.789000 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.2:0
--local_ip_for_outbound_sockets=127.20.210.2
--webserver_interface=127.20.210.2
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--builtin_ntp_servers=127.20.210.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:56:06.792703 23372 heartbeater.cc:347] Connected to a master server at 127.20.210.62:37935
I20240501 19:56:06.792878 23372 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:06.793283 23372 heartbeater.cc:511] Master 127.20.210.62:37935 requested a full tablet report, sending...
I20240501 19:56:06.794661 23184 ts_manager.cc:194] Registered new tserver with Master: ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481)
I20240501 19:56:06.796200 23184 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.1:37261
W20240501 19:56:06.885401 23375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:06.885560 23375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:06.885598 23375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:06.888384 23375 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:06.888438 23375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:06.888506 23375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.2
I20240501 19:56:06.893640 23375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.20.210.2
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--never_fsync=true
--heap_profile_path=/tmp/kudu.23375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.2
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:06.894440 23375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:06.895365 23375 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:06.903549 23380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:06.904194 23383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:06.904933 23381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:06.905695 23375 server_base.cc:1034] running on GCE node
I20240501 19:56:06.906634 23375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:06.907107 23375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:06.908263 23375 hybrid_clock.cc:648] HybridClock initialized: now 1714593366908242 us; error 34 us; skew 500 ppm
I20240501 19:56:06.909930 23375 webserver.cc:460] Webserver started at http://127.20.210.2:38791/ using document root <none> and password file <none>
I20240501 19:56:06.910476 23375 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:06.910548 23375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:06.910728 23375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:06.912261 23375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "0496e037179e4b3ca72aead781c25d5a"
format_stamp: "Formatted at 2024-05-01 19:56:06 on dist-test-slave-vgh4"
I20240501 19:56:06.912734 23375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "0496e037179e4b3ca72aead781c25d5a"
format_stamp: "Formatted at 2024-05-01 19:56:06 on dist-test-slave-vgh4"
I20240501 19:56:06.916275 23375 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 19:56:06.918191 23389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:06.919113 23375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240501 19:56:06.919212 23375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "0496e037179e4b3ca72aead781c25d5a"
format_stamp: "Formatted at 2024-05-01 19:56:06 on dist-test-slave-vgh4"
I20240501 19:56:06.919306 23375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:06.949234 23375 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:06.949820 23375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:06.949973 23375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:06.950502 23375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:06.951545 23375 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:06.951594 23375 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:06.951653 23375 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:06.951683 23375 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:06.960359 23375 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.2:38367
I20240501 19:56:06.960565 23502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.2:38367 every 8 connection(s)
I20240501 19:56:06.961167 23375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 19:56:06.964637 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 23375
I20240501 19:56:06.964744 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 19:56:06.966758 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.3:0
--local_ip_for_outbound_sockets=127.20.210.3
--webserver_interface=127.20.210.3
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--builtin_ntp_servers=127.20.210.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:56:06.970721 23503 heartbeater.cc:347] Connected to a master server at 127.20.210.62:37935
I20240501 19:56:06.970923 23503 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:06.971351 23503 heartbeater.cc:511] Master 127.20.210.62:37935 requested a full tablet report, sending...
I20240501 19:56:06.972296 23184 ts_manager.cc:194] Registered new tserver with Master: 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:06.972826 23184 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.2:56215
W20240501 19:56:07.067339 23506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:07.067514 23506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:07.067581 23506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:07.070289 23506 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:07.070351 23506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:07.070432 23506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.3
I20240501 19:56:07.073751 23506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.20.210.3
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--never_fsync=true
--heap_profile_path=/tmp/kudu.23506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.3
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:07.074425 23506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:07.075363 23506 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:07.082865 23511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:07.084017 23512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:07.084214 23514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:07.084934 23506 server_base.cc:1034] running on GCE node
I20240501 19:56:07.085218 23506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:07.085641 23506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:07.086777 23506 hybrid_clock.cc:648] HybridClock initialized: now 1714593367086757 us; error 41 us; skew 500 ppm
I20240501 19:56:07.088285 23506 webserver.cc:460] Webserver started at http://127.20.210.3:36841/ using document root <none> and password file <none>
I20240501 19:56:07.088728 23506 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:07.088783 23506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:07.088937 23506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:07.090345 23506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.090749 23506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.093588 23506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20240501 19:56:07.095458 23520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.096259 23506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:07.096351 23506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.096423 23506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:07.129657 23506 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:07.130133 23506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:07.130295 23506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:07.130708 23506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:07.131428 23506 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:07.131474 23506 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.131542 23506 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:07.131573 23506 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.138818 23506 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.3:37279
I20240501 19:56:07.138871 23633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.3:37279 every 8 connection(s)
I20240501 19:56:07.139616 23506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 19:56:07.142057 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 23506
I20240501 19:56:07.142143 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 19:56:07.144044 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.4:0
--local_ip_for_outbound_sockets=127.20.210.4
--webserver_interface=127.20.210.4
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--builtin_ntp_servers=127.20.210.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:56:07.147706 23634 heartbeater.cc:347] Connected to a master server at 127.20.210.62:37935
I20240501 19:56:07.147986 23634 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:07.148531 23634 heartbeater.cc:511] Master 127.20.210.62:37935 requested a full tablet report, sending...
I20240501 19:56:07.149464 23184 ts_manager.cc:194] Registered new tserver with Master: e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279)
I20240501 19:56:07.150076 23184 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.3:53201
W20240501 19:56:07.248546 23637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:07.248731 23637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:07.248795 23637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:07.251616 23637 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:07.251710 23637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:07.251876 23637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.4
I20240501 19:56:07.255506 23637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.4:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.20.210.4
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--never_fsync=true
--heap_profile_path=/tmp/kudu.23637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.4
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:07.256215 23637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:07.257208 23637 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:07.264935 23643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:07.265594 23645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:07.266224 23642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:07.266904 23637 server_base.cc:1034] running on GCE node
I20240501 19:56:07.267196 23637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:07.267642 23637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:07.268779 23637 hybrid_clock.cc:648] HybridClock initialized: now 1714593367268771 us; error 46 us; skew 500 ppm
I20240501 19:56:07.270226 23637 webserver.cc:460] Webserver started at http://127.20.210.4:37395/ using document root <none> and password file <none>
I20240501 19:56:07.270694 23637 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:07.270748 23637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:07.270905 23637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:07.272315 23637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "6e47806645ae47278e35600cc67d5700"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.272760 23637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "6e47806645ae47278e35600cc67d5700"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.275593 23637 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20240501 19:56:07.277371 23651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.278204 23637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20240501 19:56:07.278316 23637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "6e47806645ae47278e35600cc67d5700"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.278400 23637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:07.288028 23637 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:07.288545 23637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:07.288683 23637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:07.289156 23637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:07.289934 23637 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:07.289978 23637 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.290031 23637 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:07.290055 23637 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.297813 23637 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.4:44657
I20240501 19:56:07.297849 23764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.4:44657 every 8 connection(s)
I20240501 19:56:07.298723 23637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 19:56:07.306393 23765 heartbeater.cc:347] Connected to a master server at 127.20.210.62:37935
I20240501 19:56:07.306596 23765 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:07.307054 23765 heartbeater.cc:511] Master 127.20.210.62:37935 requested a full tablet report, sending...
I20240501 19:56:07.307866 23184 ts_manager.cc:194] Registered new tserver with Master: 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657)
I20240501 19:56:07.308313 23184 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.4:32869
I20240501 19:56:07.308955 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 23637
I20240501 19:56:07.309046 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 19:56:07.310681 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.5:0
--local_ip_for_outbound_sockets=127.20.210.5
--webserver_interface=127.20.210.5
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--builtin_ntp_servers=127.20.210.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 19:56:07.408000 23768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:07.408175 23768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:07.408241 23768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:07.410948 23768 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:07.411005 23768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:07.411094 23768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.5
I20240501 19:56:07.414521 23768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.5:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.20.210.5
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--never_fsync=true
--heap_profile_path=/tmp/kudu.23768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.5
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:07.415215 23768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:07.416183 23768 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:07.424371 23773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:07.425160 23776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:07.425848 23774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:07.426651 23768 server_base.cc:1034] running on GCE node
I20240501 19:56:07.426956 23768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:07.427429 23768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:07.428570 23768 hybrid_clock.cc:648] HybridClock initialized: now 1714593367428557 us; error 26 us; skew 500 ppm
I20240501 19:56:07.430073 23768 webserver.cc:460] Webserver started at http://127.20.210.5:37583/ using document root <none> and password file <none>
I20240501 19:56:07.430553 23768 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:07.430613 23768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:07.430781 23768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:07.432272 23768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "ee9bc706484e48fb8ff739a61cabe858"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.432710 23768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "ee9bc706484e48fb8ff739a61cabe858"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.435580 23768 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20240501 19:56:07.437345 23782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.438159 23768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:07.438269 23768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "ee9bc706484e48fb8ff739a61cabe858"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.438355 23768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:07.457659 23768 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:07.458149 23768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:07.458305 23768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:07.458734 23768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:07.459506 23768 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:07.459559 23768 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.459597 23768 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:07.459612 23768 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.468093 23768 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.5:44723
I20240501 19:56:07.468139 23895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.5:44723 every 8 connection(s)
I20240501 19:56:07.468995 23768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 19:56:07.475591 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 23768
I20240501 19:56:07.475710 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240501 19:56:07.477192 23896 heartbeater.cc:347] Connected to a master server at 127.20.210.62:37935
I20240501 19:56:07.477392 23896 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:07.477823 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.6:0
--local_ip_for_outbound_sockets=127.20.210.6
--webserver_interface=127.20.210.6
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--builtin_ntp_servers=127.20.210.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:56:07.477949 23896 heartbeater.cc:511] Master 127.20.210.62:37935 requested a full tablet report, sending...
I20240501 19:56:07.478950 23184 ts_manager.cc:194] Registered new tserver with Master: ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:07.479532 23184 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.5:59957
W20240501 19:56:07.576350 23899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:07.576505 23899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:07.576543 23899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:07.579175 23899 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:07.579226 23899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:07.579282 23899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.6
I20240501 19:56:07.582984 23899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.6:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.20.210.6
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--never_fsync=true
--heap_profile_path=/tmp/kudu.23899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.6
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:07.583644 23899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:07.584605 23899 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:07.591941 23904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:07.592670 23905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:07.593364 23907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:07.594048 23899 server_base.cc:1034] running on GCE node
I20240501 19:56:07.594354 23899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:07.594816 23899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:07.595958 23899 hybrid_clock.cc:648] HybridClock initialized: now 1714593367595943 us; error 23 us; skew 500 ppm
I20240501 19:56:07.597512 23899 webserver.cc:460] Webserver started at http://127.20.210.6:39141/ using document root <none> and password file <none>
I20240501 19:56:07.597956 23899 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:07.598006 23899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:07.598193 23899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:07.599679 23899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "4f51f982ad284c22aae01c4541a621f1"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.600076 23899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "4f51f982ad284c22aae01c4541a621f1"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.603036 23899 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 19:56:07.604890 23913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.605720 23899 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:07.605816 23899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "4f51f982ad284c22aae01c4541a621f1"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.605893 23899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:07.617638 23899 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:07.618103 23899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:07.618253 23899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:07.618685 23899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:07.619396 23899 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:07.619441 23899 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.619478 23899 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:07.619506 23899 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.627507 23899 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.6:37667
I20240501 19:56:07.627560 24026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.6:37667 every 8 connection(s)
I20240501 19:56:07.628280 23899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240501 19:56:07.633313 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 23899
I20240501 19:56:07.633421 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240501 19:56:07.635816 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.7:0
--local_ip_for_outbound_sockets=127.20.210.7
--webserver_interface=127.20.210.7
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--builtin_ntp_servers=127.20.210.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:56:07.635885 24027 heartbeater.cc:347] Connected to a master server at 127.20.210.62:37935
I20240501 19:56:07.636063 24027 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:07.636502 24027 heartbeater.cc:511] Master 127.20.210.62:37935 requested a full tablet report, sending...
I20240501 19:56:07.637404 23184 ts_manager.cc:194] Registered new tserver with Master: 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667)
I20240501 19:56:07.637964 23184 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.6:38339
W20240501 19:56:07.729619 24030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:07.729776 24030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:07.729815 24030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:07.732553 24030 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:07.732609 24030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:07.732671 24030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.7
I20240501 19:56:07.735733 24030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.7:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.20.210.7
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--never_fsync=true
--heap_profile_path=/tmp/kudu.24030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.7
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:07.736461 24030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:07.737360 24030 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:07.744884 24036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:07.745572 24038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:07.746340 24035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:07.747071 24030 server_base.cc:1034] running on GCE node
I20240501 19:56:07.747354 24030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:07.747785 24030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:07.748931 24030 hybrid_clock.cc:648] HybridClock initialized: now 1714593367748920 us; error 31 us; skew 500 ppm
I20240501 19:56:07.750533 24030 webserver.cc:460] Webserver started at http://127.20.210.7:42705/ using document root <none> and password file <none>
I20240501 19:56:07.751046 24030 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:07.751106 24030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:07.751266 24030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:07.752760 24030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "3b3f502cbeb1451087a06f95a3d93f2b"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.753233 24030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "3b3f502cbeb1451087a06f95a3d93f2b"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.756368 24030 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 19:56:07.758220 24044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.759116 24030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:07.759213 24030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "3b3f502cbeb1451087a06f95a3d93f2b"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:07.759287 24030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:07.775804 24030 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:07.776285 24030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:07.776418 24030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:07.776850 24030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:07.777637 24030 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:07.777685 24030 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.777737 24030 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:07.777767 24030 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:07.786010 24030 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.7:35019
I20240501 19:56:07.786057 24157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.7:35019 every 8 connection(s)
I20240501 19:56:07.786859 24030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 19:56:07.790782 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 24030
I20240501 19:56:07.790874 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240501 19:56:07.794795 24158 heartbeater.cc:347] Connected to a master server at 127.20.210.62:37935
I20240501 19:56:07.794966 24158 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:07.795365 24158 heartbeater.cc:511] Master 127.20.210.62:37935 requested a full tablet report, sending...
I20240501 19:56:07.796136 23184 ts_manager.cc:194] Registered new tserver with Master: 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019)
I20240501 19:56:07.796619 23184 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.7:34365
I20240501 19:56:07.798148 23372 heartbeater.cc:503] Master 127.20.210.62:37935 was elected leader, sending a full tablet report...
I20240501 19:56:07.804445 21320 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240501 19:56:07.812047 23184 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50560:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240501 19:56:07.828543 23961 tablet_service.cc:1467] Processing CreateTablet for tablet 44f96d2d427d43008630eb7bcc39caf9 (DEFAULT_TABLE table=TestTable [id=e130370886f346e289c105d0836dc0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:07.828867 23437 tablet_service.cc:1467] Processing CreateTablet for tablet 44f96d2d427d43008630eb7bcc39caf9 (DEFAULT_TABLE table=TestTable [id=e130370886f346e289c105d0836dc0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:07.829416 23961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f96d2d427d43008630eb7bcc39caf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:07.829597 23437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f96d2d427d43008630eb7bcc39caf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:07.829668 23830 tablet_service.cc:1467] Processing CreateTablet for tablet 44f96d2d427d43008630eb7bcc39caf9 (DEFAULT_TABLE table=TestTable [id=e130370886f346e289c105d0836dc0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:07.830505 23830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f96d2d427d43008630eb7bcc39caf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:07.831218 23306 tablet_service.cc:1467] Processing CreateTablet for tablet 44f96d2d427d43008630eb7bcc39caf9 (DEFAULT_TABLE table=TestTable [id=e130370886f346e289c105d0836dc0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:07.831964 23306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f96d2d427d43008630eb7bcc39caf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:07.835608 23699 tablet_service.cc:1467] Processing CreateTablet for tablet 44f96d2d427d43008630eb7bcc39caf9 (DEFAULT_TABLE table=TestTable [id=e130370886f346e289c105d0836dc0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:07.836239 23699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f96d2d427d43008630eb7bcc39caf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:07.836536 24181 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858: Bootstrap starting.
I20240501 19:56:07.837272 24182 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:07.838081 24180 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1: Bootstrap starting.
I20240501 19:56:07.838414 24181 tablet_bootstrap.cc:654] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:07.839102 24181 log.cc:824] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:07.839704 24182 tablet_bootstrap.cc:654] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:07.840029 24180 tablet_bootstrap.cc:654] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:07.840409 24182 log.cc:824] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:07.840610 24181 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858: No bootstrap required, opened a new log
I20240501 19:56:07.840775 24181 ts_tablet_manager.cc:1367] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 19:56:07.840821 24180 log.cc:824] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:07.841356 24184 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700: Bootstrap starting.
I20240501 19:56:07.841616 24187 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:07.843201 24184 tablet_bootstrap.cc:654] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:07.844013 24187 tablet_bootstrap.cc:654] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:07.844014 24184 log.cc:824] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:07.844441 24181 raft_consensus.cc:348] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:07.844640 24181 raft_consensus.cc:374] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:07.844655 24187 log.cc:824] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:07.844702 24181 raft_consensus.cc:729] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Initialized, Role: FOLLOWER
I20240501 19:56:07.845173 24181 consensus_queue.cc:260] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:07.845705 24180 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1: No bootstrap required, opened a new log
I20240501 19:56:07.845744 24184 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700: No bootstrap required, opened a new log
I20240501 19:56:07.845772 23896 heartbeater.cc:503] Master 127.20.210.62:37935 was elected leader, sending a full tablet report...
I20240501 19:56:07.845858 24180 ts_tablet_manager.cc:1367] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20240501 19:56:07.845876 24184 ts_tablet_manager.cc:1367] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240501 19:56:07.845945 24181 ts_tablet_manager.cc:1398] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240501 19:56:07.847059 24187 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:07.847198 24187 ts_tablet_manager.cc:1367] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20240501 19:56:07.848722 24180 raft_consensus.cc:348] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:07.848871 24180 raft_consensus.cc:374] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:07.848922 24180 raft_consensus.cc:729] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Initialized, Role: FOLLOWER
I20240501 19:56:07.849258 24180 consensus_queue.cc:260] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:07.850010 24180 ts_tablet_manager.cc:1398] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240501 19:56:07.850025 24187 raft_consensus.cc:348] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:07.850207 24187 raft_consensus.cc:374] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:07.850270 24187 raft_consensus.cc:729] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:07.850317 24027 heartbeater.cc:503] Master 127.20.210.62:37935 was elected leader, sending a full tablet report...
I20240501 19:56:07.850605 24182 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:07.850606 24187 consensus_queue.cc:260] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:07.850734 24182 ts_tablet_manager.cc:1367] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20240501 19:56:07.851313 24187 ts_tablet_manager.cc:1398] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240501 19:56:07.851374 23503 heartbeater.cc:503] Master 127.20.210.62:37935 was elected leader, sending a full tablet report...
I20240501 19:56:07.853909 24184 raft_consensus.cc:348] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:07.854103 24184 raft_consensus.cc:374] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:07.854154 24184 raft_consensus.cc:729] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Initialized, Role: FOLLOWER
I20240501 19:56:07.854089 24182 raft_consensus.cc:348] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:07.854297 24182 raft_consensus.cc:374] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:07.854351 24182 raft_consensus.cc:729] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:07.854527 24184 consensus_queue.cc:260] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:07.854693 24182 consensus_queue.cc:260] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:07.855228 24184 ts_tablet_manager.cc:1398] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20240501 19:56:07.855263 23765 heartbeater.cc:503] Master 127.20.210.62:37935 was elected leader, sending a full tablet report...
I20240501 19:56:07.855327 24182 ts_tablet_manager.cc:1398] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
W20240501 19:56:07.878963 24028 tablet.cc:2369] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:56:07.950760 24190 raft_consensus.cc:482] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:07.950935 24190 raft_consensus.cc:504] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:07.952127 24190 leader_election.cc:288] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481)
I20240501 19:56:07.956946 23457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f96d2d427d43008630eb7bcc39caf9" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a" is_pre_election: true
I20240501 19:56:07.957049 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f96d2d427d43008630eb7bcc39caf9" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:07.957216 23457 raft_consensus.cc:2454] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
I20240501 19:56:07.957329 23981 raft_consensus.cc:2454] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
I20240501 19:56:07.957348 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f96d2d427d43008630eb7bcc39caf9" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700" is_pre_election: true
I20240501 19:56:07.957579 23719 raft_consensus.cc:2454] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
I20240501 19:56:07.957716 23784 leader_election.cc:302] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 4f51f982ad284c22aae01c4541a621f1, ee9bc706484e48fb8ff739a61cabe858; no voters:
I20240501 19:56:07.957986 24190 raft_consensus.cc:2781] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:07.958077 24190 raft_consensus.cc:482] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:07.958170 24190 raft_consensus.cc:3037] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:07.958393 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f96d2d427d43008630eb7bcc39caf9" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd" is_pre_election: true
I20240501 19:56:07.958600 23326 raft_consensus.cc:2454] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
I20240501 19:56:07.959192 24190 raft_consensus.cc:504] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:07.959638 24190 leader_election.cc:288] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 election: Requested vote from peers 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481)
I20240501 19:56:07.959887 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f96d2d427d43008630eb7bcc39caf9" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700"
I20240501 19:56:07.959913 23457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f96d2d427d43008630eb7bcc39caf9" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a"
I20240501 19:56:07.959997 23719 raft_consensus.cc:3037] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:07.959997 23457 raft_consensus.cc:3037] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:07.959954 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f96d2d427d43008630eb7bcc39caf9" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1"
I20240501 19:56:07.960007 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f96d2d427d43008630eb7bcc39caf9" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd"
I20240501 19:56:07.960078 23981 raft_consensus.cc:3037] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:07.960098 23326 raft_consensus.cc:3037] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:07.961021 23719 raft_consensus.cc:2454] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 1.
I20240501 19:56:07.961309 23981 raft_consensus.cc:2454] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 1.
I20240501 19:56:07.961334 23326 raft_consensus.cc:2454] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 1.
I20240501 19:56:07.961743 23786 leader_election.cc:302] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 4f51f982ad284c22aae01c4541a621f1, 6e47806645ae47278e35600cc67d5700, ee9bc706484e48fb8ff739a61cabe858; no voters:
I20240501 19:56:07.961913 24190 raft_consensus.cc:2781] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:07.962126 24190 raft_consensus.cc:686] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Becoming Leader. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Running, Role: LEADER
I20240501 19:56:07.962488 23457 raft_consensus.cc:2454] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 1.
I20240501 19:56:07.962541 24190 consensus_queue.cc:237] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
W20240501 19:56:07.962612 23504 tablet.cc:2369] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:56:07.965744 23183 catalog_manager.cc:5484] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5). New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } }
W20240501 19:56:07.969884 23897 tablet.cc:2369] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:56:07.976532 21320 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240501 19:56:07.977672 21320 ts_itest-base.cc:233] Waiting for 1 tablets on tserver ad9452840e4e4184a640fb63dab754fd to finish bootstrapping
I20240501 19:56:07.982084 21320 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 0496e037179e4b3ca72aead781c25d5a to finish bootstrapping
I20240501 19:56:07.986145 21320 ts_itest-base.cc:233] Waiting for 0 tablets on tserver e8ddd74cfd954c7a8c2d284c335d7826 to finish bootstrapping
I20240501 19:56:07.990594 21320 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 6e47806645ae47278e35600cc67d5700 to finish bootstrapping
I20240501 19:56:07.994596 21320 ts_itest-base.cc:233] Waiting for 1 tablets on tserver ee9bc706484e48fb8ff739a61cabe858 to finish bootstrapping
I20240501 19:56:07.998495 21320 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 4f51f982ad284c22aae01c4541a621f1 to finish bootstrapping
I20240501 19:56:08.005731 21320 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 3b3f502cbeb1451087a06f95a3d93f2b to finish bootstrapping
I20240501 19:56:08.010485 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 24030
W20240501 19:56:08.035813 23373 tablet.cc:2369] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 19:56:08.049793 23766 tablet.cc:2369] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:56:08.152932 23634 heartbeater.cc:503] Master 127.20.210.62:37935 was elected leader, sending a full tablet report...
I20240501 19:56:08.402338 24190 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:08.476971 24190 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:08.481362 24190 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:08.481559 24210 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:11.767052 23183 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50560:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:11.774140 23699 tablet_service.cc:1467] Processing CreateTablet for tablet 107c33af3fbf4edbb3d3b4ba880e4661 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.774308 23830 tablet_service.cc:1467] Processing CreateTablet for tablet 107c33af3fbf4edbb3d3b4ba880e4661 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.774497 23699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107c33af3fbf4edbb3d3b4ba880e4661. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.774590 23830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107c33af3fbf4edbb3d3b4ba880e4661. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.775269 23437 tablet_service.cc:1467] Processing CreateTablet for tablet 107c33af3fbf4edbb3d3b4ba880e4661 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.775548 23437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107c33af3fbf4edbb3d3b4ba880e4661. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.775750 23960 tablet_service.cc:1467] Processing CreateTablet for tablet 78b919c4db7745fd87151d8834866e26 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.776016 23960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78b919c4db7745fd87151d8834866e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.776932 23961 tablet_service.cc:1467] Processing CreateTablet for tablet 87ebb3dfc00d4744960bde9a07c59e23 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.777134 23961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ebb3dfc00d4744960bde9a07c59e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.777177 23306 tablet_service.cc:1467] Processing CreateTablet for tablet 107c33af3fbf4edbb3d3b4ba880e4661 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.777442 23306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107c33af3fbf4edbb3d3b4ba880e4661. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.779273 23437 tablet_service.cc:1467] Processing CreateTablet for tablet 87ebb3dfc00d4744960bde9a07c59e23 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.779526 23437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ebb3dfc00d4744960bde9a07c59e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.780153 23830 tablet_service.cc:1467] Processing CreateTablet for tablet 78b919c4db7745fd87151d8834866e26 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.780628 23830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78b919c4db7745fd87151d8834866e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.781134 23699 tablet_service.cc:1467] Processing CreateTablet for tablet 87ebb3dfc00d4744960bde9a07c59e23 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.781374 23699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ebb3dfc00d4744960bde9a07c59e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.781452 24227 tablet_bootstrap.cc:492] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:11.782565 24227 tablet_bootstrap.cc:654] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.782670 24225 tablet_bootstrap.cc:492] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700: Bootstrap starting.
I20240501 19:56:11.782756 23567 tablet_service.cc:1467] Processing CreateTablet for tablet 87ebb3dfc00d4744960bde9a07c59e23 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.783695 23567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ebb3dfc00d4744960bde9a07c59e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.783975 24229 tablet_bootstrap.cc:492] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:11.784873 24229 tablet_bootstrap.cc:654] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.785140 24225 tablet_bootstrap.cc:654] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.785712 24227 tablet_bootstrap.cc:492] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:11.785818 24226 tablet_bootstrap.cc:492] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858: Bootstrap starting.
I20240501 19:56:11.785826 24227 ts_tablet_manager.cc:1367] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 19:56:11.786346 24227 raft_consensus.cc:348] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.786661 23566 tablet_service.cc:1467] Processing CreateTablet for tablet 78b919c4db7745fd87151d8834866e26 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.786908 23566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78b919c4db7745fd87151d8834866e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.787004 24227 raft_consensus.cc:374] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.787074 24227 raft_consensus.cc:729] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.787243 24227 consensus_queue.cc:260] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.787360 24226 tablet_bootstrap.cc:654] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.787573 24227 ts_tablet_manager.cc:1398] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:11.788326 24225 tablet_bootstrap.cc:492] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700: No bootstrap required, opened a new log
I20240501 19:56:11.788445 24225 ts_tablet_manager.cc:1367] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 19:56:11.788976 24225 raft_consensus.cc:348] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.789144 24225 raft_consensus.cc:374] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.789173 23568 tablet_service.cc:1467] Processing CreateTablet for tablet 107c33af3fbf4edbb3d3b4ba880e4661 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.789214 24225 raft_consensus.cc:729] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.789408 23568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107c33af3fbf4edbb3d3b4ba880e4661. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.789419 24225 consensus_queue.cc:260] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.789839 24225 ts_tablet_manager.cc:1398] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:11.791025 24225 tablet_bootstrap.cc:492] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700: Bootstrap starting.
I20240501 19:56:11.779301 23436 tablet_service.cc:1467] Processing CreateTablet for tablet 78b919c4db7745fd87151d8834866e26 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.791819 23436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78b919c4db7745fd87151d8834866e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.791887 24225 tablet_bootstrap.cc:654] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.791965 23306 tablet_service.cc:1467] Processing CreateTablet for tablet 87ebb3dfc00d4744960bde9a07c59e23 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.792210 23306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ebb3dfc00d4744960bde9a07c59e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.793495 24236 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:11.794340 23305 tablet_service.cc:1467] Processing CreateTablet for tablet 78b919c4db7745fd87151d8834866e26 (DEFAULT_TABLE table=rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:11.794556 23305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78b919c4db7745fd87151d8834866e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:11.794658 24227 tablet_bootstrap.cc:492] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:11.795043 24228 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1: Bootstrap starting.
I20240501 19:56:11.795483 24227 tablet_bootstrap.cc:654] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.795758 24236 tablet_bootstrap.cc:654] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.795924 24228 tablet_bootstrap.cc:654] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.796654 24227 tablet_bootstrap.cc:492] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:11.796721 24227 ts_tablet_manager.cc:1367] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:11.796941 24236 log.cc:824] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:11.797120 24227 raft_consensus.cc:348] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.797240 24227 raft_consensus.cc:374] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.797293 24227 raft_consensus.cc:729] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.797433 24227 consensus_queue.cc:260] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.797653 24229 tablet_bootstrap.cc:492] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:11.797719 24227 ts_tablet_manager.cc:1398] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:11.797762 24229 ts_tablet_manager.cc:1367] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240501 19:56:11.797911 24227 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:11.798027 24226 tablet_bootstrap.cc:492] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858: No bootstrap required, opened a new log
I20240501 19:56:11.798128 24226 ts_tablet_manager.cc:1367] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240501 19:56:11.798539 24229 raft_consensus.cc:348] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.798648 24229 raft_consensus.cc:374] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.798743 24227 tablet_bootstrap.cc:654] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.798730 24229 raft_consensus.cc:729] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.798694 24226 raft_consensus.cc:348] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.798907 24226 raft_consensus.cc:374] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.798969 24226 raft_consensus.cc:729] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.799036 24229 consensus_queue.cc:260] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.799109 24226 consensus_queue.cc:260] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.799463 24226 ts_tablet_manager.cc:1398] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:11.799624 24226 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858: Bootstrap starting.
I20240501 19:56:11.799788 24227 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:11.799872 24227 ts_tablet_manager.cc:1367] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 19:56:11.800274 24227 raft_consensus.cc:348] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.800385 24227 raft_consensus.cc:374] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.800426 24227 raft_consensus.cc:729] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.800587 24226 tablet_bootstrap.cc:654] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.800549 24227 consensus_queue.cc:260] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.800781 24227 ts_tablet_manager.cc:1398] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:11.800843 24225 tablet_bootstrap.cc:492] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700: No bootstrap required, opened a new log
I20240501 19:56:11.800923 24225 ts_tablet_manager.cc:1367] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240501 19:56:11.801358 24225 raft_consensus.cc:348] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.801473 24225 raft_consensus.cc:374] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.801453 24229 ts_tablet_manager.cc:1398] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:56:11.801517 24225 raft_consensus.cc:729] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.801641 24225 consensus_queue.cc:260] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.801929 24225 ts_tablet_manager.cc:1398] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:11.802106 24228 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1: No bootstrap required, opened a new log
I20240501 19:56:11.802186 24228 ts_tablet_manager.cc:1367] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240501 19:56:11.802726 24229 tablet_bootstrap.cc:492] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:11.802699 24228 raft_consensus.cc:348] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.802822 24228 raft_consensus.cc:374] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.802866 24228 raft_consensus.cc:729] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.802985 24228 consensus_queue.cc:260] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.803289 24228 ts_tablet_manager.cc:1398] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:11.803392 24236 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826: No bootstrap required, opened a new log
I20240501 19:56:11.803532 24236 ts_tablet_manager.cc:1367] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20240501 19:56:11.803735 24228 tablet_bootstrap.cc:492] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1: Bootstrap starting.
I20240501 19:56:11.804587 24229 tablet_bootstrap.cc:654] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.804615 24228 tablet_bootstrap.cc:654] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.804642 24233 raft_consensus.cc:482] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:11.804728 24233 raft_consensus.cc:504] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.805868 24226 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858: No bootstrap required, opened a new log
I20240501 19:56:11.805940 24226 ts_tablet_manager.cc:1367] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 19:56:11.806039 24233 leader_election.cc:288] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657)
I20240501 19:56:11.806514 24226 raft_consensus.cc:348] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.806658 24226 raft_consensus.cc:374] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.806690 24226 raft_consensus.cc:729] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.806797 24226 consensus_queue.cc:260] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.807018 24226 ts_tablet_manager.cc:1398] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:11.807256 24236 raft_consensus.cc:348] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.807466 24236 raft_consensus.cc:374] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.807523 24236 raft_consensus.cc:729] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.807960 24228 tablet_bootstrap.cc:492] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1: No bootstrap required, opened a new log
I20240501 19:56:11.807906 24236 consensus_queue.cc:260] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.808027 24228 ts_tablet_manager.cc:1367] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 19:56:11.808383 24228 raft_consensus.cc:348] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.808482 24228 raft_consensus.cc:374] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.808514 24228 raft_consensus.cc:729] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.808625 24228 consensus_queue.cc:260] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.808780 24236 ts_tablet_manager.cc:1398] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240501 19:56:11.808863 24228 ts_tablet_manager.cc:1398] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:11.809029 24229 tablet_bootstrap.cc:492] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:11.809082 24236 tablet_bootstrap.cc:492] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:11.809186 24229 ts_tablet_manager.cc:1367] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 19:56:11.809926 24236 tablet_bootstrap.cc:654] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.810317 24229 raft_consensus.cc:348] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.810423 24229 raft_consensus.cc:374] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.810459 24229 raft_consensus.cc:729] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.810585 24229 consensus_queue.cc:260] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.810858 24229 ts_tablet_manager.cc:1398] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:11.811021 24229 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:11.811851 24229 tablet_bootstrap.cc:654] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.816558 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ebb3dfc00d4744960bde9a07c59e23" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:11.816951 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ebb3dfc00d4744960bde9a07c59e23" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd" is_pre_election: true
I20240501 19:56:11.817071 23326 raft_consensus.cc:2454] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 0.
I20240501 19:56:11.817293 24236 tablet_bootstrap.cc:492] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826: No bootstrap required, opened a new log
I20240501 19:56:11.817373 24236 ts_tablet_manager.cc:1367] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
W20240501 19:56:11.817365 23393 leader_election.cc:341] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:56:11.817847 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ebb3dfc00d4744960bde9a07c59e23" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700" is_pre_election: true
I20240501 19:56:11.817852 24236 raft_consensus.cc:348] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.817945 24236 raft_consensus.cc:374] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.817950 23719 raft_consensus.cc:2454] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 0.
I20240501 19:56:11.817979 24236 raft_consensus.cc:729] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.818097 24236 consensus_queue.cc:260] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.818195 23393 leader_election.cc:302] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 3 yes votes; 1 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 6e47806645ae47278e35600cc67d5700, ad9452840e4e4184a640fb63dab754fd; no voters: e8ddd74cfd954c7a8c2d284c335d7826
I20240501 19:56:11.818457 24236 ts_tablet_manager.cc:1398] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:11.818491 24233 raft_consensus.cc:2781] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:11.818534 24233 raft_consensus.cc:482] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:11.818519 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ebb3dfc00d4744960bde9a07c59e23" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:11.818578 24233 raft_consensus.cc:3037] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:11.818630 23981 raft_consensus.cc:2454] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 0.
I20240501 19:56:11.818929 24236 tablet_bootstrap.cc:492] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:11.819744 24233 raft_consensus.cc:504] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.820053 24236 tablet_bootstrap.cc:654] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:11.820238 24233 leader_election.cc:288] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 election: Requested vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657)
I20240501 19:56:11.820914 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ebb3dfc00d4744960bde9a07c59e23" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1"
I20240501 19:56:11.821009 23981 raft_consensus.cc:3037] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:11.821496 24229 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:11.821578 24229 ts_tablet_manager.cc:1367] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240501 19:56:11.821961 23981 raft_consensus.cc:2454] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 1.
I20240501 19:56:11.822055 24229 raft_consensus.cc:348] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.822177 24229 raft_consensus.cc:374] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.822225 24229 raft_consensus.cc:729] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.822316 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ebb3dfc00d4744960bde9a07c59e23" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd"
I20240501 19:56:11.822396 24229 consensus_queue.cc:260] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.822696 24229 ts_tablet_manager.cc:1398] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:11.822728 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ebb3dfc00d4744960bde9a07c59e23" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
I20240501 19:56:11.822783 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ebb3dfc00d4744960bde9a07c59e23" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700"
I20240501 19:56:11.822881 23719 raft_consensus.cc:3037] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Advancing to term 1
W20240501 19:56:11.823287 23393 leader_election.cc:341] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:56:11.822399 23326 raft_consensus.cc:3037] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:11.823598 24236 tablet_bootstrap.cc:492] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826: No bootstrap required, opened a new log
I20240501 19:56:11.823671 24236 ts_tablet_manager.cc:1367] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240501 19:56:11.823875 23719 raft_consensus.cc:2454] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 1.
I20240501 19:56:11.824153 24210 raft_consensus.cc:482] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:11.824160 24236 raft_consensus.cc:348] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.824257 24236 raft_consensus.cc:374] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:11.824295 24236 raft_consensus.cc:729] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: FOLLOWER
I20240501 19:56:11.824250 24210 raft_consensus.cc:504] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.824358 23326 raft_consensus.cc:2454] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 1.
I20240501 19:56:11.824426 24236 consensus_queue.cc:260] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.824571 23393 leader_election.cc:302] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 3 yes votes; 1 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 4f51f982ad284c22aae01c4541a621f1, 6e47806645ae47278e35600cc67d5700; no voters: e8ddd74cfd954c7a8c2d284c335d7826
I20240501 19:56:11.824707 24236 ts_tablet_manager.cc:1398] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:11.824834 24210 leader_election.cc:288] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:11.825163 24233 raft_consensus.cc:2781] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:11.825371 24233 raft_consensus.cc:686] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Becoming Leader. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Running, Role: LEADER
I20240501 19:56:11.825522 23457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107c33af3fbf4edbb3d3b4ba880e4661" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a" is_pre_election: true
I20240501 19:56:11.825593 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107c33af3fbf4edbb3d3b4ba880e4661" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd" is_pre_election: true
I20240501 19:56:11.825613 23457 raft_consensus.cc:2454] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
I20240501 19:56:11.825625 24233 consensus_queue.cc:237] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.825686 23326 raft_consensus.cc:2454] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
I20240501 19:56:11.825951 23785 leader_election.cc:302] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, ad9452840e4e4184a640fb63dab754fd, ee9bc706484e48fb8ff739a61cabe858; no voters:
I20240501 19:56:11.826104 24210 raft_consensus.cc:2781] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:11.826154 24210 raft_consensus.cc:482] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:11.826185 24210 raft_consensus.cc:3037] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:11.827643 24210 raft_consensus.cc:504] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.827672 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107c33af3fbf4edbb3d3b4ba880e4661" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700" is_pre_election: true
I20240501 19:56:11.827773 23719 raft_consensus.cc:2454] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
I20240501 19:56:11.828130 24210 leader_election.cc:288] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 election: Requested vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:11.828025 23183 catalog_manager.cc:5484] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0496e037179e4b3ca72aead781c25d5a (127.20.210.2). New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } }
I20240501 19:56:11.828297 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107c33af3fbf4edbb3d3b4ba880e4661" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd"
I20240501 19:56:11.828426 23326 raft_consensus.cc:3037] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:11.828498 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107c33af3fbf4edbb3d3b4ba880e4661" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700"
I20240501 19:56:11.828589 23719 raft_consensus.cc:3037] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:11.829171 23326 raft_consensus.cc:2454] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 1.
I20240501 19:56:11.829345 23719 raft_consensus.cc:2454] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 1.
I20240501 19:56:11.829629 23457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107c33af3fbf4edbb3d3b4ba880e4661" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a"
I20240501 19:56:11.829723 23457 raft_consensus.cc:3037] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:11.829934 23785 leader_election.cc:302] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 6e47806645ae47278e35600cc67d5700, ad9452840e4e4184a640fb63dab754fd, ee9bc706484e48fb8ff739a61cabe858; no voters:
I20240501 19:56:11.830093 24210 raft_consensus.cc:2781] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:11.830288 24210 raft_consensus.cc:686] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Becoming Leader. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Running, Role: LEADER
I20240501 19:56:11.830425 24210 consensus_queue.cc:237] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.830565 23457 raft_consensus.cc:2454] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 1.
I20240501 19:56:11.831959 23183 catalog_manager.cc:5484] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5). New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:11.833065 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107c33af3fbf4edbb3d3b4ba880e4661" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:11.833112 23587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "107c33af3fbf4edbb3d3b4ba880e4661" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
I20240501 19:56:11.833256 23588 raft_consensus.cc:2454] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
W20240501 19:56:11.893488 23635 tablet.cc:2369] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:56:11.901980 24257 raft_consensus.cc:482] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:11.902079 24257 raft_consensus.cc:504] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.902545 24244 raft_consensus.cc:482] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:11.902652 24244 raft_consensus.cc:504] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.903394 24257 leader_election.cc:288] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:11.903734 24244 leader_election.cc:288] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723), e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:11.908869 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ebb3dfc00d4744960bde9a07c59e23" candidate_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:11.909034 23981 raft_consensus.cc:2379] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8ddd74cfd954c7a8c2d284c335d7826 in current term 1: Already voted for candidate 0496e037179e4b3ca72aead781c25d5a in this term.
I20240501 19:56:11.909030 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ebb3dfc00d4744960bde9a07c59e23" candidate_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd" is_pre_election: true
I20240501 19:56:11.909183 23326 raft_consensus.cc:2379] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8ddd74cfd954c7a8c2d284c335d7826 in current term 1: Already voted for candidate 0496e037179e4b3ca72aead781c25d5a in this term.
I20240501 19:56:11.910099 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b919c4db7745fd87151d8834866e26" candidate_uuid: "4f51f982ad284c22aae01c4541a621f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:11.910279 23588 raft_consensus.cc:2454] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f51f982ad284c22aae01c4541a621f1 in term 0.
I20240501 19:56:11.913326 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ebb3dfc00d4744960bde9a07c59e23" candidate_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700" is_pre_election: true
I20240501 19:56:11.913391 23457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ebb3dfc00d4744960bde9a07c59e23" candidate_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a" is_pre_election: true
I20240501 19:56:11.913420 23456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b919c4db7745fd87151d8834866e26" candidate_uuid: "4f51f982ad284c22aae01c4541a621f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a" is_pre_election: true
I20240501 19:56:11.913499 23719 raft_consensus.cc:2379] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8ddd74cfd954c7a8c2d284c335d7826 in current term 1: Already voted for candidate 0496e037179e4b3ca72aead781c25d5a in this term.
I20240501 19:56:11.913547 23456 raft_consensus.cc:2454] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f51f982ad284c22aae01c4541a621f1 in term 0.
I20240501 19:56:11.913800 23521 leader_election.cc:302] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 5 voters: 1 yes votes; 3 no votes. yes voters: e8ddd74cfd954c7a8c2d284c335d7826; no voters: 0496e037179e4b3ca72aead781c25d5a, 4f51f982ad284c22aae01c4541a621f1, ad9452840e4e4184a640fb63dab754fd
I20240501 19:56:11.914016 23914 leader_election.cc:302] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 4f51f982ad284c22aae01c4541a621f1, e8ddd74cfd954c7a8c2d284c335d7826; no voters:
I20240501 19:56:11.914098 24257 raft_consensus.cc:3037] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:11.914281 24244 raft_consensus.cc:2781] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:11.914345 24244 raft_consensus.cc:482] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:11.914386 24244 raft_consensus.cc:3037] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:11.914709 23850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b919c4db7745fd87151d8834866e26" candidate_uuid: "4f51f982ad284c22aae01c4541a621f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858" is_pre_election: true
I20240501 19:56:11.914938 23850 raft_consensus.cc:2454] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f51f982ad284c22aae01c4541a621f1 in term 0.
I20240501 19:56:11.915205 24257 raft_consensus.cc:2726] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 19:56:11.915303 24244 raft_consensus.cc:504] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.915840 24244 leader_election.cc:288] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [CANDIDATE]: Term 1 election: Requested vote from peers ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723), e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:11.916007 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b919c4db7745fd87151d8834866e26" candidate_uuid: "4f51f982ad284c22aae01c4541a621f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd" is_pre_election: true
I20240501 19:56:11.916057 23325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b919c4db7745fd87151d8834866e26" candidate_uuid: "4f51f982ad284c22aae01c4541a621f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd"
I20240501 19:56:11.916131 23326 raft_consensus.cc:2454] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f51f982ad284c22aae01c4541a621f1 in term 0.
I20240501 19:56:11.916132 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b919c4db7745fd87151d8834866e26" candidate_uuid: "4f51f982ad284c22aae01c4541a621f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
I20240501 19:56:11.916231 23588 raft_consensus.cc:3037] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:11.916208 23456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b919c4db7745fd87151d8834866e26" candidate_uuid: "4f51f982ad284c22aae01c4541a621f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a"
I20240501 19:56:11.916296 23456 raft_consensus.cc:3037] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:11.916318 23850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b919c4db7745fd87151d8834866e26" candidate_uuid: "4f51f982ad284c22aae01c4541a621f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858"
I20240501 19:56:11.916422 23850 raft_consensus.cc:3037] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:11.917119 23588 raft_consensus.cc:2454] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f51f982ad284c22aae01c4541a621f1 in term 1.
I20240501 19:56:11.917213 23456 raft_consensus.cc:2454] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f51f982ad284c22aae01c4541a621f1 in term 1.
I20240501 19:56:11.917415 23850 raft_consensus.cc:2454] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f51f982ad284c22aae01c4541a621f1 in term 1.
I20240501 19:56:11.917438 23914 leader_election.cc:302] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 3 yes votes; 1 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 4f51f982ad284c22aae01c4541a621f1, e8ddd74cfd954c7a8c2d284c335d7826; no voters: ad9452840e4e4184a640fb63dab754fd
I20240501 19:56:11.917594 24244 raft_consensus.cc:2781] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:11.917779 24244 raft_consensus.cc:686] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 1 LEADER]: Becoming Leader. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Running, Role: LEADER
I20240501 19:56:11.918071 24244 consensus_queue.cc:237] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:11.920501 23183 catalog_manager.cc:5484] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6). New cstate: current_term: 1 leader_uuid: "4f51f982ad284c22aae01c4541a621f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } }
W20240501 19:56:12.041199 24278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:12.041436 24278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:12.044485 24278 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240501 19:56:12.135845 23719 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.136118 23981 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.136550 24260 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.136667 23588 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.137666 23326 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.138116 24233 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.138437 24233 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.141331 23718 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.144012 23456 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.144625 24210 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.144990 23588 raft_consensus.cc:3037] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.145072 24244 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.146071 23326 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.146111 23588 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.146075 23324 raft_consensus.cc:3037] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.146585 24233 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.147675 23324 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.148088 24244 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.151209 23456 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.151453 24210 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.151571 24262 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.153708 24262 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.157178 24306 mvcc.cc:205] Tried to move back new op lower bound from 7022974452263268352 to 7022974451001315328. Current Snapshot: MvccSnapshot[applied={T|T < 7022974452263268352}]
I20240501 19:56:12.157658 23850 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.158073 24244 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.162158 24314 mvcc.cc:205] Tried to move back new op lower bound from 7022974452263268352 to 7022974451001315328. Current Snapshot: MvccSnapshot[applied={T|T < 7022974452263268352}]
I20240501 19:56:12.164073 23586 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.164433 24244 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.173744 24316 mvcc.cc:205] Tried to move back new op lower bound from 7022974452263268352 to 7022974451001315328. Current Snapshot: MvccSnapshot[applied={T|T < 7022974452263268352}]
I20240501 19:56:12.174590 24312 mvcc.cc:205] Tried to move back new op lower bound from 7022974452263268352 to 7022974451001315328. Current Snapshot: MvccSnapshot[applied={T|T < 7022974452263268352}]
I20240501 19:56:12.190599 24310 mvcc.cc:205] Tried to move back new op lower bound from 7022974452303794176 to 7022974451019636736. Current Snapshot: MvccSnapshot[applied={T|T < 7022974452279283712}]
I20240501 19:56:12.204653 24341 mvcc.cc:205] Tried to move back new op lower bound from 7022974452296990720 to 7022974451379224576. Current Snapshot: MvccSnapshot[applied={T|T < 7022974452296990720}]
INSERT report
rows total: 2000
time total: 164.004 ms
time per row: 0.0820022 ms
I20240501 19:56:12.253300 23178 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50560:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:12.258394 23567 tablet_service.cc:1467] Processing CreateTablet for tablet b2eea4673ba94b0fa5a2e0f099532777 (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.258559 23305 tablet_service.cc:1467] Processing CreateTablet for tablet b2eea4673ba94b0fa5a2e0f099532777 (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.258667 23567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2eea4673ba94b0fa5a2e0f099532777. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.258775 23305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2eea4673ba94b0fa5a2e0f099532777. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.259704 23306 tablet_service.cc:1467] Processing CreateTablet for tablet af8e2ce72fb24a18b54b0d2e49383b41 (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.259902 23306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af8e2ce72fb24a18b54b0d2e49383b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.259861 23961 tablet_service.cc:1467] Processing CreateTablet for tablet b2eea4673ba94b0fa5a2e0f099532777 (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.260084 23961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2eea4673ba94b0fa5a2e0f099532777. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.260699 24229 tablet_bootstrap.cc:492] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:12.260740 24236 tablet_bootstrap.cc:492] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:12.261422 24229 tablet_bootstrap.cc:654] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.261488 24236 tablet_bootstrap.cc:654] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.261976 24228 tablet_bootstrap.cc:492] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1: Bootstrap starting.
I20240501 19:56:12.262228 23436 tablet_service.cc:1467] Processing CreateTablet for tablet 292aa245734c41d9814b8347d543b8fb (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.262498 23436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292aa245734c41d9814b8347d543b8fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.262580 24236 tablet_bootstrap.cc:492] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826: No bootstrap required, opened a new log
I20240501 19:56:12.262651 24236 ts_tablet_manager.cc:1367] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:56:12.262871 24228 tablet_bootstrap.cc:654] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.263064 23960 tablet_service.cc:1467] Processing CreateTablet for tablet af8e2ce72fb24a18b54b0d2e49383b41 (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.263113 24236 raft_consensus.cc:348] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.263226 24236 raft_consensus.cc:374] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.263273 24236 raft_consensus.cc:729] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.263285 23960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af8e2ce72fb24a18b54b0d2e49383b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.263388 24236 consensus_queue.cc:260] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.263655 24236 ts_tablet_manager.cc:1398] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 19:56:12.263844 24228 tablet_bootstrap.cc:492] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1: No bootstrap required, opened a new log
I20240501 19:56:12.263912 24228 ts_tablet_manager.cc:1367] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:12.264173 24228 raft_consensus.cc:348] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.264242 24228 raft_consensus.cc:374] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.264261 24228 raft_consensus.cc:729] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.264375 24228 consensus_queue.cc:260] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.264523 24227 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:12.264628 24228 ts_tablet_manager.cc:1398] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:12.265071 24228 tablet_bootstrap.cc:492] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1: Bootstrap starting.
I20240501 19:56:12.265259 24227 tablet_bootstrap.cc:654] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.265697 24228 tablet_bootstrap.cc:654] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.266343 23829 tablet_service.cc:1467] Processing CreateTablet for tablet 292aa245734c41d9814b8347d543b8fb (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.266606 23829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292aa245734c41d9814b8347d543b8fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.266798 23830 tablet_service.cc:1467] Processing CreateTablet for tablet b2eea4673ba94b0fa5a2e0f099532777 (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.266762 23699 tablet_service.cc:1467] Processing CreateTablet for tablet b2eea4673ba94b0fa5a2e0f099532777 (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.266781 23697 tablet_service.cc:1467] Processing CreateTablet for tablet af8e2ce72fb24a18b54b0d2e49383b41 (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.267028 23830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2eea4673ba94b0fa5a2e0f099532777. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.267077 23699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2eea4673ba94b0fa5a2e0f099532777. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.267220 23697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af8e2ce72fb24a18b54b0d2e49383b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.267465 23698 tablet_service.cc:1467] Processing CreateTablet for tablet 292aa245734c41d9814b8347d543b8fb (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.267685 23698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292aa245734c41d9814b8347d543b8fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.269415 24228 tablet_bootstrap.cc:492] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1: No bootstrap required, opened a new log
I20240501 19:56:12.269467 24228 ts_tablet_manager.cc:1367] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 19:56:12.269517 24225 tablet_bootstrap.cc:492] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700: Bootstrap starting.
I20240501 19:56:12.269582 24226 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858: Bootstrap starting.
I20240501 19:56:12.269780 24228 raft_consensus.cc:348] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } }
I20240501 19:56:12.269850 24228 raft_consensus.cc:374] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.269867 24228 raft_consensus.cc:729] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.269937 24228 consensus_queue.cc:260] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } }
I20240501 19:56:12.270130 24228 ts_tablet_manager.cc:1398] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:12.270169 24225 tablet_bootstrap.cc:654] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.270179 24226 tablet_bootstrap.cc:654] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.270646 23567 tablet_service.cc:1467] Processing CreateTablet for tablet 292aa245734c41d9814b8347d543b8fb (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.270794 23568 tablet_service.cc:1467] Processing CreateTablet for tablet af8e2ce72fb24a18b54b0d2e49383b41 (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.270870 23567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292aa245734c41d9814b8347d543b8fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.271014 23568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af8e2ce72fb24a18b54b0d2e49383b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.271073 24226 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858: No bootstrap required, opened a new log
I20240501 19:56:12.271137 24226 ts_tablet_manager.cc:1367] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:12.271523 24226 raft_consensus.cc:348] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } }
I20240501 19:56:12.271620 24226 raft_consensus.cc:374] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.271651 24226 raft_consensus.cc:729] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.271764 24226 consensus_queue.cc:260] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } }
I20240501 19:56:12.272048 24226 ts_tablet_manager.cc:1398] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:12.272207 24226 tablet_bootstrap.cc:492] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858: Bootstrap starting.
I20240501 19:56:12.273010 24226 tablet_bootstrap.cc:654] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.273010 24236 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:12.273036 23828 tablet_service.cc:1467] Processing CreateTablet for tablet af8e2ce72fb24a18b54b0d2e49383b41 (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.273258 23828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af8e2ce72fb24a18b54b0d2e49383b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.273535 24229 tablet_bootstrap.cc:492] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:12.273634 24229 ts_tablet_manager.cc:1367] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.001s
I20240501 19:56:12.273990 24229 raft_consensus.cc:348] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.274063 24229 raft_consensus.cc:374] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.274083 24229 raft_consensus.cc:729] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.274180 24229 consensus_queue.cc:260] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.274462 24229 ts_tablet_manager.cc:1398] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:12.274636 24229 tablet_bootstrap.cc:492] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:12.274955 24236 tablet_bootstrap.cc:654] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.275405 24226 tablet_bootstrap.cc:492] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858: No bootstrap required, opened a new log
I20240501 19:56:12.275481 24226 ts_tablet_manager.cc:1367] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:12.275518 24229 tablet_bootstrap.cc:654] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.275864 24226 raft_consensus.cc:348] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.275956 24226 raft_consensus.cc:374] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.275981 24226 raft_consensus.cc:729] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.276094 24226 consensus_queue.cc:260] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.276559 23961 tablet_service.cc:1467] Processing CreateTablet for tablet 292aa245734c41d9814b8347d543b8fb (DEFAULT_TABLE table=rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.276773 23961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292aa245734c41d9814b8347d543b8fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.276815 24226 ts_tablet_manager.cc:1398] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:12.277107 24229 tablet_bootstrap.cc:492] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:12.277161 24229 ts_tablet_manager.cc:1367] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:12.277460 24229 raft_consensus.cc:348] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } }
I20240501 19:56:12.277563 24229 raft_consensus.cc:374] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.277588 24229 raft_consensus.cc:729] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.277707 24229 consensus_queue.cc:260] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } }
I20240501 19:56:12.278153 24229 ts_tablet_manager.cc:1398] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:12.276950 24226 tablet_bootstrap.cc:492] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858: Bootstrap starting.
I20240501 19:56:12.279585 24228 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1: Bootstrap starting.
I20240501 19:56:12.280356 24225 tablet_bootstrap.cc:492] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700: No bootstrap required, opened a new log
I20240501 19:56:12.280405 24228 tablet_bootstrap.cc:654] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.280438 24225 ts_tablet_manager.cc:1367] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240501 19:56:12.280592 24227 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:12.280678 24227 ts_tablet_manager.cc:1367] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.001s
I20240501 19:56:12.280978 24225 raft_consensus.cc:348] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } }
I20240501 19:56:12.281103 24225 raft_consensus.cc:374] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.281042 24227 raft_consensus.cc:348] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } }
I20240501 19:56:12.281142 24227 raft_consensus.cc:374] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.281136 24225 raft_consensus.cc:729] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.281193 24227 raft_consensus.cc:729] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.281273 24225 consensus_queue.cc:260] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } }
I20240501 19:56:12.281314 24227 consensus_queue.cc:260] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } }
I20240501 19:56:12.281463 24228 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1: No bootstrap required, opened a new log
I20240501 19:56:12.281522 24228 ts_tablet_manager.cc:1367] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:12.281560 24225 ts_tablet_manager.cc:1398] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:12.281584 24227 ts_tablet_manager.cc:1398] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:12.281713 24225 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700: Bootstrap starting.
I20240501 19:56:12.281858 24228 raft_consensus.cc:348] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } }
I20240501 19:56:12.281965 24228 raft_consensus.cc:374] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.282011 24228 raft_consensus.cc:729] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.282111 24228 consensus_queue.cc:260] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } }
I20240501 19:56:12.282222 24210 raft_consensus.cc:482] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:12.282315 24210 raft_consensus.cc:504] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.282371 24228 ts_tablet_manager.cc:1398] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:12.282872 24210 leader_election.cc:288] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657)
I20240501 19:56:12.283131 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2eea4673ba94b0fa5a2e0f099532777" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:12.283139 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2eea4673ba94b0fa5a2e0f099532777" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700" is_pre_election: true
I20240501 19:56:12.283255 23588 raft_consensus.cc:2454] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
I20240501 19:56:12.283465 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2eea4673ba94b0fa5a2e0f099532777" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:12.283574 23981 raft_consensus.cc:2454] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
I20240501 19:56:12.283536 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2eea4673ba94b0fa5a2e0f099532777" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd" is_pre_election: true
I20240501 19:56:12.283664 23326 raft_consensus.cc:2454] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
W20240501 19:56:12.283984 23786 leader_election.cc:341] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:56:12.284258 23784 leader_election.cc:302] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 3 yes votes; 1 no votes. yes voters: 4f51f982ad284c22aae01c4541a621f1, e8ddd74cfd954c7a8c2d284c335d7826, ee9bc706484e48fb8ff739a61cabe858; no voters: 6e47806645ae47278e35600cc67d5700
I20240501 19:56:12.284489 24210 raft_consensus.cc:2781] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:12.284545 24210 raft_consensus.cc:482] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:12.284579 24210 raft_consensus.cc:3037] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.285238 24225 tablet_bootstrap.cc:654] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.285379 24210 raft_consensus.cc:504] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.285631 24226 tablet_bootstrap.cc:654] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.286285 24210 leader_election.cc:288] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 election: Requested vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657)
I20240501 19:56:12.286518 24225 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700: No bootstrap required, opened a new log
I20240501 19:56:12.286485 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2eea4673ba94b0fa5a2e0f099532777" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd"
I20240501 19:56:12.286589 23326 raft_consensus.cc:3037] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.286602 24225 ts_tablet_manager.cc:1367] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 19:56:12.286862 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2eea4673ba94b0fa5a2e0f099532777" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700"
I20240501 19:56:12.286998 24225 raft_consensus.cc:348] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } }
I20240501 19:56:12.287110 24225 raft_consensus.cc:374] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.287145 24225 raft_consensus.cc:729] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Initialized, Role: FOLLOWER
W20240501 19:56:12.287158 23786 leader_election.cc:341] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:56:12.287264 24225 consensus_queue.cc:260] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } }
I20240501 19:56:12.287464 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2eea4673ba94b0fa5a2e0f099532777" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1"
I20240501 19:56:12.287510 23326 raft_consensus.cc:2454] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 1.
I20240501 19:56:12.287557 23981 raft_consensus.cc:3037] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.287629 24225 ts_tablet_manager.cc:1398] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:12.287704 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2eea4673ba94b0fa5a2e0f099532777" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
I20240501 19:56:12.287776 24225 tablet_bootstrap.cc:492] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700: Bootstrap starting.
I20240501 19:56:12.287797 23588 raft_consensus.cc:3037] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.288426 23981 raft_consensus.cc:2454] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 1.
I20240501 19:56:12.288604 24225 tablet_bootstrap.cc:654] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.288637 23588 raft_consensus.cc:2454] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 1.
I20240501 19:56:12.288902 23786 leader_election.cc:302] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 3 yes votes; 1 no votes. yes voters: 4f51f982ad284c22aae01c4541a621f1, e8ddd74cfd954c7a8c2d284c335d7826, ee9bc706484e48fb8ff739a61cabe858; no voters: 6e47806645ae47278e35600cc67d5700
I20240501 19:56:12.289100 24210 raft_consensus.cc:2781] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:12.289175 24210 raft_consensus.cc:686] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Becoming Leader. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Running, Role: LEADER
I20240501 19:56:12.289289 24210 consensus_queue.cc:237] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.289769 24225 tablet_bootstrap.cc:492] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700: No bootstrap required, opened a new log
I20240501 19:56:12.289844 24225 ts_tablet_manager.cc:1367] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:12.290163 24226 tablet_bootstrap.cc:492] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858: No bootstrap required, opened a new log
I20240501 19:56:12.290282 24226 ts_tablet_manager.cc:1367] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240501 19:56:12.290257 24225 raft_consensus.cc:348] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.290367 24225 raft_consensus.cc:374] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.290413 24225 raft_consensus.cc:729] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.290515 24225 consensus_queue.cc:260] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.290676 24226 raft_consensus.cc:348] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } }
I20240501 19:56:12.290760 24225 ts_tablet_manager.cc:1398] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:12.290778 24226 raft_consensus.cc:374] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.290807 24226 raft_consensus.cc:729] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.290938 24226 consensus_queue.cc:260] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } }
I20240501 19:56:12.291198 24226 ts_tablet_manager.cc:1398] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:12.292016 24236 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826: No bootstrap required, opened a new log
I20240501 19:56:12.292104 24236 ts_tablet_manager.cc:1367] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.001s
I20240501 19:56:12.292460 24236 raft_consensus.cc:348] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } }
I20240501 19:56:12.292552 24236 raft_consensus.cc:374] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.292587 24236 raft_consensus.cc:729] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.292733 24236 consensus_queue.cc:260] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } }
I20240501 19:56:12.292994 24236 ts_tablet_manager.cc:1398] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 19:56:12.293197 24236 tablet_bootstrap.cc:492] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:12.294103 24236 tablet_bootstrap.cc:654] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.295061 23184 catalog_manager.cc:5484] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5). New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } }
I20240501 19:56:12.295390 24236 tablet_bootstrap.cc:492] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826: No bootstrap required, opened a new log
I20240501 19:56:12.295475 24236 ts_tablet_manager.cc:1367] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:12.295879 24236 raft_consensus.cc:348] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } }
I20240501 19:56:12.296000 24236 raft_consensus.cc:374] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.296033 24236 raft_consensus.cc:729] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.296168 24236 consensus_queue.cc:260] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } }
I20240501 19:56:12.296458 24236 ts_tablet_manager.cc:1398] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:12.339641 24257 raft_consensus.cc:482] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:12.339774 24257 raft_consensus.cc:504] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } }
I20240501 19:56:12.340389 24257 leader_election.cc:288] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667)
I20240501 19:56:12.340606 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af8e2ce72fb24a18b54b0d2e49383b41" candidate_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700" is_pre_election: true
I20240501 19:56:12.340754 23719 raft_consensus.cc:2454] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8ddd74cfd954c7a8c2d284c335d7826 in term 0.
I20240501 19:56:12.340764 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af8e2ce72fb24a18b54b0d2e49383b41" candidate_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd" is_pre_election: true
I20240501 19:56:12.340929 23326 raft_consensus.cc:2454] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8ddd74cfd954c7a8c2d284c335d7826 in term 0.
I20240501 19:56:12.341288 23523 leader_election.cc:302] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 6e47806645ae47278e35600cc67d5700, ad9452840e4e4184a640fb63dab754fd, e8ddd74cfd954c7a8c2d284c335d7826; no voters:
I20240501 19:56:12.341369 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af8e2ce72fb24a18b54b0d2e49383b41" candidate_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:12.341455 24257 raft_consensus.cc:2781] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:12.341506 24257 raft_consensus.cc:482] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:12.341497 23981 raft_consensus.cc:2454] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8ddd74cfd954c7a8c2d284c335d7826 in term 0.
I20240501 19:56:12.341544 24257 raft_consensus.cc:3037] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.342644 24257 raft_consensus.cc:504] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } }
I20240501 19:56:12.343132 24257 leader_election.cc:288] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [CANDIDATE]: Term 1 election: Requested vote from peers 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667)
I20240501 19:56:12.343353 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af8e2ce72fb24a18b54b0d2e49383b41" candidate_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700"
I20240501 19:56:12.343461 23719 raft_consensus.cc:3037] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.343500 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af8e2ce72fb24a18b54b0d2e49383b41" candidate_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1"
I20240501 19:56:12.343550 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af8e2ce72fb24a18b54b0d2e49383b41" candidate_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd"
I20240501 19:56:12.343595 23981 raft_consensus.cc:3037] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.343642 23326 raft_consensus.cc:3037] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.344460 23719 raft_consensus.cc:2454] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8ddd74cfd954c7a8c2d284c335d7826 in term 1.
I20240501 19:56:12.344550 23326 raft_consensus.cc:2454] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8ddd74cfd954c7a8c2d284c335d7826 in term 1.
I20240501 19:56:12.344719 23981 raft_consensus.cc:2454] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8ddd74cfd954c7a8c2d284c335d7826 in term 1.
I20240501 19:56:12.344802 23523 leader_election.cc:302] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 6e47806645ae47278e35600cc67d5700, ad9452840e4e4184a640fb63dab754fd, e8ddd74cfd954c7a8c2d284c335d7826; no voters:
I20240501 19:56:12.344978 24257 raft_consensus.cc:2781] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:12.345165 24257 raft_consensus.cc:686] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 LEADER]: Becoming Leader. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Running, Role: LEADER
I20240501 19:56:12.345381 23849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af8e2ce72fb24a18b54b0d2e49383b41" candidate_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858"
I20240501 19:56:12.345355 23850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af8e2ce72fb24a18b54b0d2e49383b41" candidate_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858" is_pre_election: true
I20240501 19:56:12.345503 23849 raft_consensus.cc:3037] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.345490 24257 consensus_queue.cc:237] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } }
I20240501 19:56:12.346450 23849 raft_consensus.cc:2454] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8ddd74cfd954c7a8c2d284c335d7826 in term 1.
I20240501 19:56:12.347751 23184 catalog_manager.cc:5484] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } }
I20240501 19:56:12.359194 24235 raft_consensus.cc:482] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:12.359280 24235 raft_consensus.cc:504] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:12.360239 24235 leader_election.cc:288] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:12.364904 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2eea4673ba94b0fa5a2e0f099532777" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd" is_pre_election: true
I20240501 19:56:12.364930 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2eea4673ba94b0fa5a2e0f099532777" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:12.365056 23326 raft_consensus.cc:2379] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e47806645ae47278e35600cc67d5700 in current term 1: Already voted for candidate ee9bc706484e48fb8ff739a61cabe858 in this term.
I20240501 19:56:12.365069 23588 raft_consensus.cc:2379] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e47806645ae47278e35600cc67d5700 in current term 1: Already voted for candidate ee9bc706484e48fb8ff739a61cabe858 in this term.
I20240501 19:56:12.365351 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2eea4673ba94b0fa5a2e0f099532777" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:12.365491 23981 raft_consensus.cc:2379] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6e47806645ae47278e35600cc67d5700 in current term 1: Already voted for candidate ee9bc706484e48fb8ff739a61cabe858 in this term.
I20240501 19:56:12.365727 23653 leader_election.cc:302] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 5 voters: 1 yes votes; 3 no votes. yes voters: 6e47806645ae47278e35600cc67d5700; no voters: 4f51f982ad284c22aae01c4541a621f1, ad9452840e4e4184a640fb63dab754fd, e8ddd74cfd954c7a8c2d284c335d7826
I20240501 19:56:12.365931 24235 raft_consensus.cc:3037] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.365942 23849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2eea4673ba94b0fa5a2e0f099532777" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858" is_pre_election: true
I20240501 19:56:12.366755 24235 raft_consensus.cc:2726] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 19:56:12.582616 24235 raft_consensus.cc:482] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:12.582746 24235 raft_consensus.cc:504] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } }
I20240501 19:56:12.583312 24235 leader_election.cc:288] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:12.583580 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292aa245734c41d9814b8347d543b8fb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:12.583667 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292aa245734c41d9814b8347d543b8fb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:12.583724 23981 raft_consensus.cc:2454] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:12.583825 23588 raft_consensus.cc:2454] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:12.584205 23653 leader_election.cc:302] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 4f51f982ad284c22aae01c4541a621f1, 6e47806645ae47278e35600cc67d5700, e8ddd74cfd954c7a8c2d284c335d7826; no voters:
I20240501 19:56:12.584386 23849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292aa245734c41d9814b8347d543b8fb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858" is_pre_election: true
I20240501 19:56:12.584457 24235 raft_consensus.cc:2781] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:12.584508 24235 raft_consensus.cc:482] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:12.584512 23849 raft_consensus.cc:2454] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:12.584543 24235 raft_consensus.cc:3037] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.585546 24235 raft_consensus.cc:504] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } }
I20240501 19:56:12.586010 24235 leader_election.cc:288] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 election: Requested vote from peers 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:12.586398 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292aa245734c41d9814b8347d543b8fb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
I20240501 19:56:12.586400 23849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292aa245734c41d9814b8347d543b8fb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858"
I20240501 19:56:12.586524 23588 raft_consensus.cc:3037] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.586524 23849 raft_consensus.cc:3037] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.587411 23588 raft_consensus.cc:2454] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:12.587411 23849 raft_consensus.cc:2454] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:12.587780 23652 leader_election.cc:302] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 6e47806645ae47278e35600cc67d5700, e8ddd74cfd954c7a8c2d284c335d7826, ee9bc706484e48fb8ff739a61cabe858; no voters:
I20240501 19:56:12.587921 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292aa245734c41d9814b8347d543b8fb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1"
I20240501 19:56:12.588027 23981 raft_consensus.cc:3037] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.588091 24235 raft_consensus.cc:2781] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:12.588258 24235 raft_consensus.cc:686] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Becoming Leader. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Running, Role: LEADER
I20240501 19:56:12.588565 24235 consensus_queue.cc:237] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } }
I20240501 19:56:12.588950 23981 raft_consensus.cc:2454] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:12.589640 23456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292aa245734c41d9814b8347d543b8fb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a" is_pre_election: true
I20240501 19:56:12.589654 23457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292aa245734c41d9814b8347d543b8fb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a"
I20240501 19:56:12.589777 23456 raft_consensus.cc:2454] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:12.591511 23184 catalog_manager.cc:5484] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e47806645ae47278e35600cc67d5700 (127.20.210.4). New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } }
I20240501 19:56:12.679138 24210 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.697074 24210 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.700858 24392 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.701059 24262 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.728343 24257 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 19:56:12.752472 24388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:12.752671 24388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:12.755528 24388 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240501 19:56:12.794719 24257 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.803231 24380 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.808069 24257 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.846710 23456 raft_consensus.cc:3037] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:12.851104 23849 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.851259 23588 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.851630 24235 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.852180 24235 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.853170 23980 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.853591 24235 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:12.898972 23456 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:12.899504 24235 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 128.153 ms
time per row: 0.0640763 ms
I20240501 19:56:12.930987 23184 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50560:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:12.937589 23828 tablet_service.cc:1467] Processing CreateTablet for tablet 5898d010acd94c35a73a6e5c28f8d313 (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.937641 23567 tablet_service.cc:1467] Processing CreateTablet for tablet 5898d010acd94c35a73a6e5c28f8d313 (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.937863 23567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5898d010acd94c35a73a6e5c28f8d313. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.937858 23828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5898d010acd94c35a73a6e5c28f8d313. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.938551 23436 tablet_service.cc:1467] Processing CreateTablet for tablet 5898d010acd94c35a73a6e5c28f8d313 (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.938797 23436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5898d010acd94c35a73a6e5c28f8d313. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.939047 23304 tablet_service.cc:1467] Processing CreateTablet for tablet 1c9623f516f84445ade4bf06eb05673f (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.939231 23305 tablet_service.cc:1467] Processing CreateTablet for tablet 9f0625a730b0408b963d94f24e25fed7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.939330 23304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c9623f516f84445ade4bf06eb05673f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.939438 23305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f0625a730b0408b963d94f24e25fed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.940472 23961 tablet_service.cc:1467] Processing CreateTablet for tablet 9f0625a730b0408b963d94f24e25fed7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.940718 23961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f0625a730b0408b963d94f24e25fed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.940964 23699 tablet_service.cc:1467] Processing CreateTablet for tablet 5898d010acd94c35a73a6e5c28f8d313 (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.941193 23699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5898d010acd94c35a73a6e5c28f8d313. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.942472 23698 tablet_service.cc:1467] Processing CreateTablet for tablet 9f0625a730b0408b963d94f24e25fed7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.942641 23698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f0625a730b0408b963d94f24e25fed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.942888 24467 tablet_bootstrap.cc:492] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1: Bootstrap starting.
I20240501 19:56:12.943465 24466 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:12.943718 24467 tablet_bootstrap.cc:654] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.944350 23436 tablet_service.cc:1467] Processing CreateTablet for tablet 9f0625a730b0408b963d94f24e25fed7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.944355 23437 tablet_service.cc:1467] Processing CreateTablet for tablet 1c9623f516f84445ade4bf06eb05673f (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.944594 23436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f0625a730b0408b963d94f24e25fed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.944764 23437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c9623f516f84445ade4bf06eb05673f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.945075 23828 tablet_service.cc:1467] Processing CreateTablet for tablet 1c9623f516f84445ade4bf06eb05673f (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.945293 23828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c9623f516f84445ade4bf06eb05673f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.945502 23567 tablet_service.cc:1467] Processing CreateTablet for tablet 9f0625a730b0408b963d94f24e25fed7 (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.945711 23567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f0625a730b0408b963d94f24e25fed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.947274 24466 tablet_bootstrap.cc:654] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.947274 23568 tablet_service.cc:1467] Processing CreateTablet for tablet 1c9623f516f84445ade4bf06eb05673f (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.939047 23306 tablet_service.cc:1467] Processing CreateTablet for tablet 5898d010acd94c35a73a6e5c28f8d313 (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.947487 23568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c9623f516f84445ade4bf06eb05673f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.947587 23306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5898d010acd94c35a73a6e5c28f8d313. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.948210 24466 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:12.948288 24466 ts_tablet_manager.cc:1367] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 19:56:12.948582 24464 tablet_bootstrap.cc:492] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858: Bootstrap starting.
I20240501 19:56:12.948921 24466 raft_consensus.cc:348] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } }
I20240501 19:56:12.949501 24464 tablet_bootstrap.cc:654] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.949676 24465 tablet_bootstrap.cc:492] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:12.949743 24466 raft_consensus.cc:374] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.949805 24466 raft_consensus.cc:729] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.949918 24466 consensus_queue.cc:260] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } }
I20240501 19:56:12.950198 24466 ts_tablet_manager.cc:1398] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:12.950384 24466 tablet_bootstrap.cc:492] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:12.951053 24465 tablet_bootstrap.cc:654] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.951208 24466 tablet_bootstrap.cc:654] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.952119 24470 tablet_bootstrap.cc:492] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700: Bootstrap starting.
I20240501 19:56:12.952215 23961 tablet_service.cc:1467] Processing CreateTablet for tablet 1c9623f516f84445ade4bf06eb05673f (DEFAULT_TABLE table=rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:12.952438 23961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c9623f516f84445ade4bf06eb05673f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:12.952872 24470 tablet_bootstrap.cc:654] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.952936 24464 tablet_bootstrap.cc:492] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858: No bootstrap required, opened a new log
I20240501 19:56:12.953014 24464 ts_tablet_manager.cc:1367] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240501 19:56:12.953500 24468 tablet_bootstrap.cc:492] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:12.953472 24464 raft_consensus.cc:348] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:12.953577 24464 raft_consensus.cc:374] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.953610 24464 raft_consensus.cc:729] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.953724 24464 consensus_queue.cc:260] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:12.953914 24470 tablet_bootstrap.cc:492] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700: No bootstrap required, opened a new log
I20240501 19:56:12.953984 24470 ts_tablet_manager.cc:1367] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:12.954416 24464 ts_tablet_manager.cc:1398] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:12.954458 24470 raft_consensus.cc:348] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:12.954557 24470 raft_consensus.cc:374] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.954574 24464 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858: Bootstrap starting.
I20240501 19:56:12.954597 24470 raft_consensus.cc:729] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.954712 24470 consensus_queue.cc:260] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:12.955379 24468 tablet_bootstrap.cc:654] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.955688 24464 tablet_bootstrap.cc:654] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.956472 24468 tablet_bootstrap.cc:492] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826: No bootstrap required, opened a new log
I20240501 19:56:12.956548 24468 ts_tablet_manager.cc:1367] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:12.956774 24470 ts_tablet_manager.cc:1398] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:12.957024 24468 raft_consensus.cc:348] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:12.957132 24468 raft_consensus.cc:374] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.957163 24468 raft_consensus.cc:729] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.957299 24465 tablet_bootstrap.cc:492] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:12.957381 24465 ts_tablet_manager.cc:1367] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240501 19:56:12.957326 24468 consensus_queue.cc:260] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:12.957489 24464 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858: No bootstrap required, opened a new log
I20240501 19:56:12.957554 24464 ts_tablet_manager.cc:1367] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:12.957720 24465 raft_consensus.cc:348] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:12.957832 24465 raft_consensus.cc:374] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.957868 24465 raft_consensus.cc:729] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.957953 24464 raft_consensus.cc:348] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } }
I20240501 19:56:12.957976 24465 consensus_queue.cc:260] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:12.958055 24464 raft_consensus.cc:374] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.958086 24464 raft_consensus.cc:729] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.958185 24464 consensus_queue.cc:260] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } }
I20240501 19:56:12.958460 24464 ts_tablet_manager.cc:1398] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:12.958762 24468 ts_tablet_manager.cc:1398] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:12.958904 24468 tablet_bootstrap.cc:492] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:12.959537 24465 ts_tablet_manager.cc:1398] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:12.959709 24465 tablet_bootstrap.cc:492] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:12.959787 24470 tablet_bootstrap.cc:492] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700: Bootstrap starting.
I20240501 19:56:12.960449 24465 tablet_bootstrap.cc:654] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.960603 24470 tablet_bootstrap.cc:654] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.961254 24468 tablet_bootstrap.cc:654] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.961509 24465 tablet_bootstrap.cc:492] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:12.961577 24465 ts_tablet_manager.cc:1367] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:56:12.962003 24465 raft_consensus.cc:348] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:12.962137 24465 raft_consensus.cc:374] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.962177 24465 raft_consensus.cc:729] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.962306 24465 consensus_queue.cc:260] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:12.962383 24470 tablet_bootstrap.cc:492] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700: No bootstrap required, opened a new log
I20240501 19:56:12.962447 24470 ts_tablet_manager.cc:1367] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 19:56:12.962563 24465 ts_tablet_manager.cc:1398] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:12.962625 24466 tablet_bootstrap.cc:492] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:12.962706 24465 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:12.962698 24466 ts_tablet_manager.cc:1367] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.001s
I20240501 19:56:12.962800 24470 raft_consensus.cc:348] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:12.962898 24470 raft_consensus.cc:374] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.962926 24470 raft_consensus.cc:729] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.963037 24470 consensus_queue.cc:260] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:12.963258 24470 ts_tablet_manager.cc:1398] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:12.963232 24466 raft_consensus.cc:348] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:12.963361 24466 raft_consensus.cc:374] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.963392 24466 raft_consensus.cc:729] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.963512 24466 consensus_queue.cc:260] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:12.963693 24465 tablet_bootstrap.cc:654] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.963851 24466 ts_tablet_manager.cc:1398] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:12.963999 24466 tablet_bootstrap.cc:492] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:12.964623 24467 tablet_bootstrap.cc:492] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1: No bootstrap required, opened a new log
I20240501 19:56:12.964709 24467 ts_tablet_manager.cc:1367] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240501 19:56:12.965224 24467 raft_consensus.cc:348] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:12.965346 24467 raft_consensus.cc:374] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.965376 24467 raft_consensus.cc:729] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.965499 24467 consensus_queue.cc:260] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:12.965757 24467 ts_tablet_manager.cc:1398] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:12.965912 24467 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1: Bootstrap starting.
I20240501 19:56:12.966017 24466 tablet_bootstrap.cc:654] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.966730 24467 tablet_bootstrap.cc:654] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.967036 24465 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:12.967094 24465 ts_tablet_manager.cc:1367] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240501 19:56:12.967610 24465 raft_consensus.cc:348] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } }
I20240501 19:56:12.967708 24465 raft_consensus.cc:374] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.967739 24465 raft_consensus.cc:729] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.967783 24467 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1: No bootstrap required, opened a new log
I20240501 19:56:12.967850 24467 ts_tablet_manager.cc:1367] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:12.967875 24465 consensus_queue.cc:260] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } }
I20240501 19:56:12.968231 24467 raft_consensus.cc:348] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } }
I20240501 19:56:12.968329 24467 raft_consensus.cc:374] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.968364 24467 raft_consensus.cc:729] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.968493 24467 consensus_queue.cc:260] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } }
I20240501 19:56:12.968812 24467 ts_tablet_manager.cc:1398] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:12.969543 24465 ts_tablet_manager.cc:1398] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:56:12.970503 24468 tablet_bootstrap.cc:492] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826: No bootstrap required, opened a new log
I20240501 19:56:12.970700 24468 ts_tablet_manager.cc:1367] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240501 19:56:12.971136 24468 raft_consensus.cc:348] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:12.971264 24468 raft_consensus.cc:374] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.971302 24468 raft_consensus.cc:729] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.971406 24468 consensus_queue.cc:260] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:12.971663 24468 ts_tablet_manager.cc:1398] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:12.971823 24468 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:12.972422 24468 tablet_bootstrap.cc:654] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:12.973393 24468 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826: No bootstrap required, opened a new log
I20240501 19:56:12.973464 24468 ts_tablet_manager.cc:1367] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:12.973644 24466 tablet_bootstrap.cc:492] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:12.973709 24466 ts_tablet_manager.cc:1367] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240501 19:56:12.973834 24468 raft_consensus.cc:348] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } }
I20240501 19:56:12.973915 24468 raft_consensus.cc:374] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.973942 24468 raft_consensus.cc:729] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.974125 24466 raft_consensus.cc:348] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:12.974123 24468 consensus_queue.cc:260] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } }
I20240501 19:56:12.974205 24466 raft_consensus.cc:374] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:12.974251 24466 raft_consensus.cc:729] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:12.974368 24466 consensus_queue.cc:260] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:12.974473 24468 ts_tablet_manager.cc:1398] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:12.974611 24466 ts_tablet_manager.cc:1398] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.009208 24386 raft_consensus.cc:482] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:13.009368 24386 raft_consensus.cc:504] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.010092 24386 leader_election.cc:288] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:13.010347 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f0625a730b0408b963d94f24e25fed7" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:13.010387 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f0625a730b0408b963d94f24e25fed7" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd" is_pre_election: true
I20240501 19:56:13.010499 23588 raft_consensus.cc:2454] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:13.010499 23326 raft_consensus.cc:2454] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:13.010954 23456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f0625a730b0408b963d94f24e25fed7" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a" is_pre_election: true
I20240501 19:56:13.010979 23980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f0625a730b0408b963d94f24e25fed7" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:13.011102 23456 raft_consensus.cc:2454] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:13.011126 23980 raft_consensus.cc:2454] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:13.011432 23652 leader_election.cc:302] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 6e47806645ae47278e35600cc67d5700, ad9452840e4e4184a640fb63dab754fd; no voters:
I20240501 19:56:13.011603 24386 raft_consensus.cc:2781] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:13.011654 24386 raft_consensus.cc:482] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:13.011705 24386 raft_consensus.cc:3037] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.012722 24386 raft_consensus.cc:504] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.013197 24386 leader_election.cc:288] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 election: Requested vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:13.013492 23980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f0625a730b0408b963d94f24e25fed7" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1"
I20240501 19:56:13.013587 23980 raft_consensus.cc:3037] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.013832 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f0625a730b0408b963d94f24e25fed7" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd"
I20240501 19:56:13.013826 23456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f0625a730b0408b963d94f24e25fed7" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a"
I20240501 19:56:13.013923 23326 raft_consensus.cc:3037] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.013932 23456 raft_consensus.cc:3037] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.014511 23980 raft_consensus.cc:2454] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:13.014715 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f0625a730b0408b963d94f24e25fed7" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
I20240501 19:56:13.014760 23456 raft_consensus.cc:2454] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:13.014839 23588 raft_consensus.cc:3037] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.014910 23326 raft_consensus.cc:2454] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:13.015157 23654 leader_election.cc:302] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 6e47806645ae47278e35600cc67d5700, ad9452840e4e4184a640fb63dab754fd; no voters:
I20240501 19:56:13.015300 24386 raft_consensus.cc:2781] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:13.015370 24386 raft_consensus.cc:686] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Becoming Leader. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Running, Role: LEADER
I20240501 19:56:13.015492 24386 consensus_queue.cc:237] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.015681 23588 raft_consensus.cc:2454] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:13.017207 23177 catalog_manager.cc:5484] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e47806645ae47278e35600cc67d5700 (127.20.210.4). New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:13.047106 24386 raft_consensus.cc:482] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:13.047204 24386 raft_consensus.cc:504] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.047847 23456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898d010acd94c35a73a6e5c28f8d313" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a" is_pre_election: true
I20240501 19:56:13.047890 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898d010acd94c35a73a6e5c28f8d313" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:13.047880 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898d010acd94c35a73a6e5c28f8d313" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd" is_pre_election: true
I20240501 19:56:13.047974 23456 raft_consensus.cc:2454] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:13.047983 23588 raft_consensus.cc:2454] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:13.047995 23326 raft_consensus.cc:2454] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:13.048029 23850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898d010acd94c35a73a6e5c28f8d313" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858" is_pre_election: true
I20240501 19:56:13.048146 23850 raft_consensus.cc:2454] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:13.048207 24386 leader_election.cc:288] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481)
I20240501 19:56:13.048246 23654 leader_election.cc:302] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 6e47806645ae47278e35600cc67d5700, ad9452840e4e4184a640fb63dab754fd; no voters:
I20240501 19:56:13.048473 24386 raft_consensus.cc:2781] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:13.048532 24386 raft_consensus.cc:482] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:13.048568 24386 raft_consensus.cc:3037] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.049222 24386 raft_consensus.cc:504] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.049629 24386 leader_election.cc:288] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 election: Requested vote from peers 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481)
I20240501 19:56:13.049875 23456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898d010acd94c35a73a6e5c28f8d313" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a"
I20240501 19:56:13.049894 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898d010acd94c35a73a6e5c28f8d313" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
I20240501 19:56:13.049942 23850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898d010acd94c35a73a6e5c28f8d313" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858"
I20240501 19:56:13.049958 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898d010acd94c35a73a6e5c28f8d313" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd"
I20240501 19:56:13.049981 23456 raft_consensus.cc:3037] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.049995 23588 raft_consensus.cc:3037] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.050027 23850 raft_consensus.cc:3037] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.050041 23326 raft_consensus.cc:3037] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.050884 23326 raft_consensus.cc:2454] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:13.050876 23588 raft_consensus.cc:2454] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:13.050904 23456 raft_consensus.cc:2454] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:13.050935 23850 raft_consensus.cc:2454] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:13.051240 23655 leader_election.cc:302] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 6e47806645ae47278e35600cc67d5700, e8ddd74cfd954c7a8c2d284c335d7826, ee9bc706484e48fb8ff739a61cabe858; no voters:
I20240501 19:56:13.051663 24386 raft_consensus.cc:2781] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:13.051761 24386 raft_consensus.cc:686] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Becoming Leader. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Running, Role: LEADER
I20240501 19:56:13.051892 24386 consensus_queue.cc:237] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.053658 23177 catalog_manager.cc:5484] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e47806645ae47278e35600cc67d5700 (127.20.210.4). New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:13.065275 24395 raft_consensus.cc:482] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:13.065384 24395 raft_consensus.cc:504] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } }
I20240501 19:56:13.066430 24395 leader_election.cc:288] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723), e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667)
I20240501 19:56:13.071141 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c9623f516f84445ade4bf06eb05673f" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:13.071280 23588 raft_consensus.cc:2454] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 0.
I20240501 19:56:13.071290 23850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c9623f516f84445ade4bf06eb05673f" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858" is_pre_election: true
I20240501 19:56:13.071310 23456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c9623f516f84445ade4bf06eb05673f" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a" is_pre_election: true
I20240501 19:56:13.071398 23850 raft_consensus.cc:2454] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 0.
I20240501 19:56:13.071466 23456 raft_consensus.cc:2454] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 0.
I20240501 19:56:13.071625 23980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c9623f516f84445ade4bf06eb05673f" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:13.071666 23259 leader_election.cc:302] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: ad9452840e4e4184a640fb63dab754fd, e8ddd74cfd954c7a8c2d284c335d7826, ee9bc706484e48fb8ff739a61cabe858; no voters:
I20240501 19:56:13.071779 23980 raft_consensus.cc:2454] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 0.
I20240501 19:56:13.071882 24395 raft_consensus.cc:2781] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:13.071938 24395 raft_consensus.cc:482] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:13.071987 24395 raft_consensus.cc:3037] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.072772 24395 raft_consensus.cc:504] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } }
I20240501 19:56:13.073233 24395 leader_election.cc:288] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 election: Requested vote from peers ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723), e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667)
I20240501 19:56:13.073490 23850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c9623f516f84445ade4bf06eb05673f" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858"
I20240501 19:56:13.073506 23588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c9623f516f84445ade4bf06eb05673f" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
I20240501 19:56:13.073586 23850 raft_consensus.cc:3037] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.073607 23588 raft_consensus.cc:3037] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.073657 23980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c9623f516f84445ade4bf06eb05673f" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1"
I20240501 19:56:13.073736 23980 raft_consensus.cc:3037] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.073848 23456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c9623f516f84445ade4bf06eb05673f" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a"
I20240501 19:56:13.073931 23456 raft_consensus.cc:3037] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.074491 23850 raft_consensus.cc:2454] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 1.
I20240501 19:56:13.074569 23980 raft_consensus.cc:2454] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 1.
I20240501 19:56:13.074559 23588 raft_consensus.cc:2454] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 1.
I20240501 19:56:13.074936 23456 raft_consensus.cc:2454] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 1.
I20240501 19:56:13.075145 23259 leader_election.cc:302] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: ad9452840e4e4184a640fb63dab754fd, e8ddd74cfd954c7a8c2d284c335d7826, ee9bc706484e48fb8ff739a61cabe858; no voters:
I20240501 19:56:13.075333 24395 raft_consensus.cc:2781] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:13.075421 24395 raft_consensus.cc:686] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Becoming Leader. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Running, Role: LEADER
I20240501 19:56:13.075697 24395 consensus_queue.cc:237] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } }
I20240501 19:56:13.077994 23177 catalog_manager.cc:5484] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd reported cstate change: term changed from 0 to 1, leader changed from <none> to ad9452840e4e4184a640fb63dab754fd (127.20.210.1). New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } }
W20240501 19:56:13.217339 24494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:13.217572 24494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:13.220567 24494 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240501 19:56:13.295866 23980 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.295920 23588 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.296314 23326 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.296336 24235 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.296751 24386 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.297029 24386 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.298029 23456 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.298117 23850 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.303726 23456 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.303757 24323 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.304170 24323 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.304426 24235 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.304790 23980 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.305217 23586 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.305737 24323 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.307341 24520 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.308697 23587 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.309165 24235 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.315464 23850 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.315650 23455 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.315830 24386 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.315968 23324 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.321045 24438 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.321883 24438 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 156.616 ms
time per row: 0.0783079 ms
I20240501 19:56:13.415419 23175 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50560:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:13.420635 23961 tablet_service.cc:1467] Processing CreateTablet for tablet b1a0cebcdad24e34abc11eef9d4d891d (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.420858 23699 tablet_service.cc:1467] Processing CreateTablet for tablet b1a0cebcdad24e34abc11eef9d4d891d (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.420935 23961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1a0cebcdad24e34abc11eef9d4d891d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.421106 23699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1a0cebcdad24e34abc11eef9d4d891d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.421234 23960 tablet_service.cc:1467] Processing CreateTablet for tablet a4ef2fdbb40e48b18025136adb39defc (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.421433 23960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ef2fdbb40e48b18025136adb39defc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.422961 23437 tablet_service.cc:1467] Processing CreateTablet for tablet b1a0cebcdad24e34abc11eef9d4d891d (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.423115 23829 tablet_service.cc:1467] Processing CreateTablet for tablet 6f549fed054d433381ac6d38d64c74bb (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.423189 23437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1a0cebcdad24e34abc11eef9d4d891d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.423195 24470 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700: Bootstrap starting.
I20240501 19:56:13.423374 23829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f549fed054d433381ac6d38d64c74bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.423390 23828 tablet_service.cc:1467] Processing CreateTablet for tablet b1a0cebcdad24e34abc11eef9d4d891d (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.423560 23828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1a0cebcdad24e34abc11eef9d4d891d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.424031 24470 tablet_bootstrap.cc:654] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.425134 24465 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:13.425194 24467 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1: Bootstrap starting.
I20240501 19:56:13.425606 24464 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858: Bootstrap starting.
I20240501 19:56:13.425931 24467 tablet_bootstrap.cc:654] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.425930 24465 tablet_bootstrap.cc:654] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.426415 24464 tablet_bootstrap.cc:654] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.426584 23830 tablet_service.cc:1467] Processing CreateTablet for tablet a4ef2fdbb40e48b18025136adb39defc (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.426798 23830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ef2fdbb40e48b18025136adb39defc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.427016 23437 tablet_service.cc:1467] Processing CreateTablet for tablet a4ef2fdbb40e48b18025136adb39defc (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.427106 23436 tablet_service.cc:1467] Processing CreateTablet for tablet 6f549fed054d433381ac6d38d64c74bb (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.427197 23437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ef2fdbb40e48b18025136adb39defc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.427354 23436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f549fed054d433381ac6d38d64c74bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.427863 23568 tablet_service.cc:1467] Processing CreateTablet for tablet a4ef2fdbb40e48b18025136adb39defc (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.428112 23568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ef2fdbb40e48b18025136adb39defc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.429031 23305 tablet_service.cc:1467] Processing CreateTablet for tablet b1a0cebcdad24e34abc11eef9d4d891d (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.429271 23305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1a0cebcdad24e34abc11eef9d4d891d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.430114 24464 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858: No bootstrap required, opened a new log
I20240501 19:56:13.430189 24464 ts_tablet_manager.cc:1367] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240501 19:56:13.430385 24468 tablet_bootstrap.cc:492] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:13.430624 24464 raft_consensus.cc:348] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.430716 24464 raft_consensus.cc:374] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.430750 24464 raft_consensus.cc:729] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.430864 24464 consensus_queue.cc:260] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.431146 24464 ts_tablet_manager.cc:1398] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.431195 24466 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:13.431197 24468 tablet_bootstrap.cc:654] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.431289 24464 tablet_bootstrap.cc:492] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858: Bootstrap starting.
I20240501 19:56:13.431304 23699 tablet_service.cc:1467] Processing CreateTablet for tablet a4ef2fdbb40e48b18025136adb39defc (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.431529 23699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ef2fdbb40e48b18025136adb39defc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.431787 23306 tablet_service.cc:1467] Processing CreateTablet for tablet 6f549fed054d433381ac6d38d64c74bb (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.431962 24466 tablet_bootstrap.cc:654] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.431996 24464 tablet_bootstrap.cc:654] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.432015 23306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f549fed054d433381ac6d38d64c74bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.432739 23698 tablet_service.cc:1467] Processing CreateTablet for tablet 6f549fed054d433381ac6d38d64c74bb (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.432966 23698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f549fed054d433381ac6d38d64c74bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.434427 24468 tablet_bootstrap.cc:492] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826: No bootstrap required, opened a new log
I20240501 19:56:13.434509 24468 ts_tablet_manager.cc:1367] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240501 19:56:13.434931 24466 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:13.434891 24468 raft_consensus.cc:348] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.434995 24468 raft_consensus.cc:374] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.435006 24466 ts_tablet_manager.cc:1367] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240501 19:56:13.435027 24468 raft_consensus.cc:729] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.435124 23961 tablet_service.cc:1467] Processing CreateTablet for tablet 6f549fed054d433381ac6d38d64c74bb (DEFAULT_TABLE table=rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.435149 24468 consensus_queue.cc:260] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.435308 23961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f549fed054d433381ac6d38d64c74bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.435413 24468 ts_tablet_manager.cc:1398] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:13.435453 24467 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1: No bootstrap required, opened a new log
I20240501 19:56:13.435529 24467 ts_tablet_manager.cc:1367] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240501 19:56:13.435442 24466 raft_consensus.cc:348] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.435568 24466 raft_consensus.cc:374] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.435601 24466 raft_consensus.cc:729] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.435712 24466 consensus_queue.cc:260] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.435863 24467 raft_consensus.cc:348] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.435966 24466 ts_tablet_manager.cc:1398] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 19:56:13.435984 24467 raft_consensus.cc:374] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.436018 24467 raft_consensus.cc:729] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.436143 24466 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:13.436127 24467 consensus_queue.cc:260] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.436404 24467 ts_tablet_manager.cc:1398] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.436578 24467 tablet_bootstrap.cc:492] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1: Bootstrap starting.
I20240501 19:56:13.436960 24466 tablet_bootstrap.cc:654] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.437436 24467 tablet_bootstrap.cc:654] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.438095 24520 raft_consensus.cc:482] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:13.438195 24520 raft_consensus.cc:504] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.438899 24520 leader_election.cc:288] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:13.439219 23849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1a0cebcdad24e34abc11eef9d4d891d" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858" is_pre_election: true
I20240501 19:56:13.439355 23849 raft_consensus.cc:2454] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 0.
I20240501 19:56:13.439520 24465 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:13.439544 23455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1a0cebcdad24e34abc11eef9d4d891d" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a" is_pre_election: true
I20240501 19:56:13.439610 24465 ts_tablet_manager.cc:1367] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.001s
W20240501 19:56:13.440008 23259 leader_election.cc:341] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:56:13.440032 24465 raft_consensus.cc:348] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.440152 24465 raft_consensus.cc:374] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.440184 24465 raft_consensus.cc:729] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.440294 24465 consensus_queue.cc:260] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.440572 24465 ts_tablet_manager.cc:1398] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:13.440732 24465 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:13.441634 24470 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700: No bootstrap required, opened a new log
I20240501 19:56:13.441721 24470 ts_tablet_manager.cc:1367] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20240501 19:56:13.441884 24465 tablet_bootstrap.cc:654] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.442055 24464 tablet_bootstrap.cc:492] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858: No bootstrap required, opened a new log
I20240501 19:56:13.442144 24464 ts_tablet_manager.cc:1367] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240501 19:56:13.442245 24467 tablet_bootstrap.cc:492] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1: No bootstrap required, opened a new log
I20240501 19:56:13.442360 24467 ts_tablet_manager.cc:1367] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 19:56:13.442335 24470 raft_consensus.cc:348] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.442441 24470 raft_consensus.cc:374] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.442474 24470 raft_consensus.cc:729] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.442587 24470 consensus_queue.cc:260] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.442595 24464 raft_consensus.cc:348] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.442687 24464 raft_consensus.cc:374] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.442719 24464 raft_consensus.cc:729] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.442683 24467 raft_consensus.cc:348] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.442754 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1a0cebcdad24e34abc11eef9d4d891d" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:13.442775 24467 raft_consensus.cc:374] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.442844 24467 raft_consensus.cc:729] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.442848 24470 ts_tablet_manager.cc:1398] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.442864 23981 raft_consensus.cc:2454] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 0.
I20240501 19:56:13.442842 24464 consensus_queue.cc:260] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.443017 24470 tablet_bootstrap.cc:492] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700: Bootstrap starting.
I20240501 19:56:13.442984 24467 consensus_queue.cc:260] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.443116 24464 ts_tablet_manager.cc:1398] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.443270 24464 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858: Bootstrap starting.
I20240501 19:56:13.443313 24467 ts_tablet_manager.cc:1398] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.443435 24467 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1: Bootstrap starting.
I20240501 19:56:13.443857 24470 tablet_bootstrap.cc:654] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.444098 24464 tablet_bootstrap.cc:654] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.444244 24467 tablet_bootstrap.cc:654] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.445158 24464 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858: No bootstrap required, opened a new log
I20240501 19:56:13.445237 24464 ts_tablet_manager.cc:1367] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:13.445655 24464 raft_consensus.cc:348] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.445734 24464 raft_consensus.cc:374] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.445753 24464 raft_consensus.cc:729] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.445874 24464 consensus_queue.cc:260] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.446113 24464 ts_tablet_manager.cc:1398] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.446138 24466 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:13.446208 24466 ts_tablet_manager.cc:1367] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20240501 19:56:13.446673 24466 raft_consensus.cc:348] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.446760 24466 raft_consensus.cc:374] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.446784 24466 raft_consensus.cc:729] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.446782 24470 tablet_bootstrap.cc:492] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700: No bootstrap required, opened a new log
I20240501 19:56:13.446782 24467 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1: No bootstrap required, opened a new log
I20240501 19:56:13.446844 24467 ts_tablet_manager.cc:1367] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:13.446844 24470 ts_tablet_manager.cc:1367] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:56:13.446874 24466 consensus_queue.cc:260] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.447113 24466 ts_tablet_manager.cc:1398] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.447180 24467 raft_consensus.cc:348] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.447266 24467 raft_consensus.cc:374] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.447295 24467 raft_consensus.cc:729] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.447358 23260 leader_election.cc:302] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 3 yes votes; 1 no votes. yes voters: 4f51f982ad284c22aae01c4541a621f1, ad9452840e4e4184a640fb63dab754fd, ee9bc706484e48fb8ff739a61cabe858; no voters: 0496e037179e4b3ca72aead781c25d5a
I20240501 19:56:13.447425 24467 consensus_queue.cc:260] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.447494 24520 raft_consensus.cc:2781] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:13.447535 24520 raft_consensus.cc:482] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:13.447575 24520 raft_consensus.cc:3037] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.447719 24467 ts_tablet_manager.cc:1398] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.448272 24470 raft_consensus.cc:348] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.448376 24470 raft_consensus.cc:374] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.448408 24470 raft_consensus.cc:729] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.448395 24520 raft_consensus.cc:504] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.448532 24470 consensus_queue.cc:260] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.448762 24470 ts_tablet_manager.cc:1398] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:13.448834 24520 leader_election.cc:288] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 election: Requested vote from peers 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:13.448904 24470 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700: Bootstrap starting.
I20240501 19:56:13.449280 23455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1a0cebcdad24e34abc11eef9d4d891d" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a"
I20240501 19:56:13.449362 23455 raft_consensus.cc:3037] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.449487 23849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1a0cebcdad24e34abc11eef9d4d891d" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858"
I20240501 19:56:13.449584 23849 raft_consensus.cc:3037] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.449623 24470 tablet_bootstrap.cc:654] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.450307 23455 raft_consensus.cc:2454] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 1.
I20240501 19:56:13.451170 23849 raft_consensus.cc:2454] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 1.
I20240501 19:56:13.451186 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1a0cebcdad24e34abc11eef9d4d891d" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1"
I20240501 19:56:13.451267 23981 raft_consensus.cc:3037] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.451444 23259 leader_election.cc:302] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, ad9452840e4e4184a640fb63dab754fd, ee9bc706484e48fb8ff739a61cabe858; no voters:
I20240501 19:56:13.451575 24520 raft_consensus.cc:2781] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:13.451627 24520 raft_consensus.cc:686] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Becoming Leader. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Running, Role: LEADER
I20240501 19:56:13.451741 24520 consensus_queue.cc:237] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.452076 23981 raft_consensus.cc:2454] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 1.
I20240501 19:56:13.452845 23719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1a0cebcdad24e34abc11eef9d4d891d" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700" is_pre_election: true
I20240501 19:56:13.452942 23719 raft_consensus.cc:2454] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 0.
I20240501 19:56:13.453158 23718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1a0cebcdad24e34abc11eef9d4d891d" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700"
I20240501 19:56:13.453260 23718 raft_consensus.cc:3037] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.454123 23718 raft_consensus.cc:2454] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 1.
I20240501 19:56:13.454746 24465 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:13.454888 24465 ts_tablet_manager.cc:1367] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.001s
I20240501 19:56:13.454946 23180 catalog_manager.cc:5484] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd reported cstate change: term changed from 0 to 1, leader changed from <none> to ad9452840e4e4184a640fb63dab754fd (127.20.210.1). New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:13.455272 24470 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700: No bootstrap required, opened a new log
I20240501 19:56:13.455329 24470 ts_tablet_manager.cc:1367] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 19:56:13.455641 24470 raft_consensus.cc:348] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.455739 24470 raft_consensus.cc:374] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.455766 24470 raft_consensus.cc:729] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.455775 24465 raft_consensus.cc:348] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.455873 24465 raft_consensus.cc:374] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.455909 24465 raft_consensus.cc:729] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.455889 24470 consensus_queue.cc:260] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.456040 24465 consensus_queue.cc:260] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.456164 24470 ts_tablet_manager.cc:1398] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.456730 24465 ts_tablet_manager.cc:1398] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:13.456912 24465 tablet_bootstrap.cc:492] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:13.457687 24465 tablet_bootstrap.cc:654] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.458837 24465 tablet_bootstrap.cc:492] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:13.458899 24465 ts_tablet_manager.cc:1367] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:13.459259 24465 raft_consensus.cc:348] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.459340 24465 raft_consensus.cc:374] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.459359 24465 raft_consensus.cc:729] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.459466 24465 consensus_queue.cc:260] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.459662 24465 ts_tablet_manager.cc:1398] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:13.530050 24311 raft_consensus.cc:482] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:13.530164 24311 raft_consensus.cc:504] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.530853 24311 leader_election.cc:288] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:13.531113 23718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ef2fdbb40e48b18025136adb39defc" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700" is_pre_election: true
I20240501 19:56:13.531183 23587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ef2fdbb40e48b18025136adb39defc" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:13.531265 23718 raft_consensus.cc:2454] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 0.
I20240501 19:56:13.531256 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ef2fdbb40e48b18025136adb39defc" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:13.531320 23587 raft_consensus.cc:2454] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 0.
I20240501 19:56:13.531368 23981 raft_consensus.cc:2454] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 0.
I20240501 19:56:13.531611 23393 leader_election.cc:302] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 6e47806645ae47278e35600cc67d5700, e8ddd74cfd954c7a8c2d284c335d7826; no voters:
I20240501 19:56:13.531790 24311 raft_consensus.cc:2781] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:13.531849 24311 raft_consensus.cc:482] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:13.531883 24311 raft_consensus.cc:3037] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.532999 24311 raft_consensus.cc:504] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.533504 24311 leader_election.cc:288] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 election: Requested vote from peers 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:13.533687 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ef2fdbb40e48b18025136adb39defc" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1"
I20240501 19:56:13.533723 23587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ef2fdbb40e48b18025136adb39defc" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
I20240501 19:56:13.533787 23981 raft_consensus.cc:3037] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.533829 23587 raft_consensus.cc:3037] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.533794 23718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ef2fdbb40e48b18025136adb39defc" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700"
I20240501 19:56:13.533913 23718 raft_consensus.cc:3037] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.534786 23981 raft_consensus.cc:2454] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 1.
I20240501 19:56:13.534919 23718 raft_consensus.cc:2454] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 1.
I20240501 19:56:13.535080 23587 raft_consensus.cc:2454] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 1.
I20240501 19:56:13.535171 23393 leader_election.cc:302] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 4f51f982ad284c22aae01c4541a621f1, 6e47806645ae47278e35600cc67d5700; no voters:
I20240501 19:56:13.535352 24311 raft_consensus.cc:2781] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:13.535444 24311 raft_consensus.cc:686] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Becoming Leader. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Running, Role: LEADER
I20240501 19:56:13.535607 24311 consensus_queue.cc:237] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.535715 23849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ef2fdbb40e48b18025136adb39defc" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858" is_pre_election: true
I20240501 19:56:13.535742 23850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ef2fdbb40e48b18025136adb39defc" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858"
I20240501 19:56:13.535832 23849 raft_consensus.cc:2454] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 0.
I20240501 19:56:13.537220 23175 catalog_manager.cc:5484] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0496e037179e4b3ca72aead781c25d5a (127.20.210.2). New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:13.541359 24527 raft_consensus.cc:482] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:13.541466 24527 raft_consensus.cc:504] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.541909 24527 leader_election.cc:288] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:13.542207 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f549fed054d433381ac6d38d64c74bb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:13.542207 23850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f549fed054d433381ac6d38d64c74bb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858" is_pre_election: true
I20240501 19:56:13.542304 23455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f549fed054d433381ac6d38d64c74bb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a" is_pre_election: true
I20240501 19:56:13.542335 23981 raft_consensus.cc:2454] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:13.542346 23850 raft_consensus.cc:2454] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:13.542306 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f549fed054d433381ac6d38d64c74bb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd" is_pre_election: true
I20240501 19:56:13.542408 23455 raft_consensus.cc:2454] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:13.542438 23326 raft_consensus.cc:2454] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 0.
I20240501 19:56:13.542636 23653 leader_election.cc:302] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 4f51f982ad284c22aae01c4541a621f1, 6e47806645ae47278e35600cc67d5700, ee9bc706484e48fb8ff739a61cabe858; no voters:
I20240501 19:56:13.542809 24527 raft_consensus.cc:2781] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:13.542872 24527 raft_consensus.cc:482] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:13.542908 24527 raft_consensus.cc:3037] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.543548 24527 raft_consensus.cc:504] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.544030 24527 leader_election.cc:288] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 election: Requested vote from peers ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:13.544170 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f549fed054d433381ac6d38d64c74bb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd"
I20240501 19:56:13.544267 23326 raft_consensus.cc:3037] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.544267 23850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f549fed054d433381ac6d38d64c74bb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858"
I20240501 19:56:13.544363 23455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f549fed054d433381ac6d38d64c74bb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a"
I20240501 19:56:13.544395 23850 raft_consensus.cc:3037] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.544452 23455 raft_consensus.cc:3037] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.544503 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f549fed054d433381ac6d38d64c74bb" candidate_uuid: "6e47806645ae47278e35600cc67d5700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1"
I20240501 19:56:13.544593 23981 raft_consensus.cc:3037] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.545204 23455 raft_consensus.cc:2454] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:13.545342 23850 raft_consensus.cc:2454] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:13.545404 23981 raft_consensus.cc:2454] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
I20240501 19:56:13.545596 23652 leader_election.cc:302] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 6e47806645ae47278e35600cc67d5700, ee9bc706484e48fb8ff739a61cabe858; no voters:
I20240501 19:56:13.545749 24527 raft_consensus.cc:2781] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:13.545827 24527 raft_consensus.cc:686] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Becoming Leader. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Running, Role: LEADER
I20240501 19:56:13.545953 24527 consensus_queue.cc:237] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.547662 23175 catalog_manager.cc:5484] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e47806645ae47278e35600cc67d5700 (127.20.210.4). New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:13.575031 23326 raft_consensus.cc:2454] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e47806645ae47278e35600cc67d5700 in term 1.
W20240501 19:56:13.655362 24563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:13.655576 24563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:13.658330 24563 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240501 19:56:13.730067 23981 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.730391 23850 raft_consensus.cc:3037] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.730432 23718 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.730602 24592 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.731479 23850 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.733637 23587 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.737285 23326 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.737550 23850 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.738286 23455 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.738729 24519 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.739019 24594 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.739030 24540 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.739682 23718 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.740094 23980 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.740170 23850 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.740171 23457 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.740768 24592 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.740901 24311 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.740919 24527 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.741385 24520 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.741485 24519 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.742079 24520 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.742702 24540 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:13.750373 23981 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:13.760830 24527 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 131.805 ms
time per row: 0.0659026 ms
I20240501 19:56:13.834527 23184 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50560:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:13.839979 23568 tablet_service.cc:1467] Processing CreateTablet for tablet a4f98ae45e914fa3981237af149b0666 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.840277 23568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4f98ae45e914fa3981237af149b0666. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.840838 23306 tablet_service.cc:1467] Processing CreateTablet for tablet a4f98ae45e914fa3981237af149b0666 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.841131 23306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4f98ae45e914fa3981237af149b0666. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.842002 23829 tablet_service.cc:1467] Processing CreateTablet for tablet a4f98ae45e914fa3981237af149b0666 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.842305 23829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4f98ae45e914fa3981237af149b0666. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.842484 23830 tablet_service.cc:1467] Processing CreateTablet for tablet f53fd7f6fb694eb89eb611497e064ab5 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.842684 23830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53fd7f6fb694eb89eb611497e064ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.843436 23305 tablet_service.cc:1467] Processing CreateTablet for tablet f701e0b045d540698ba3879954d8b171 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.843676 23305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f701e0b045d540698ba3879954d8b171. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.844484 24466 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:13.844556 24468 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:13.845337 24466 tablet_bootstrap.cc:654] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.845345 24468 tablet_bootstrap.cc:654] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.845866 24464 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858: Bootstrap starting.
I20240501 19:56:13.845852 23961 tablet_service.cc:1467] Processing CreateTablet for tablet f701e0b045d540698ba3879954d8b171 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.846078 23961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f701e0b045d540698ba3879954d8b171. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.846473 23960 tablet_service.cc:1467] Processing CreateTablet for tablet f53fd7f6fb694eb89eb611497e064ab5 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.846630 24464 tablet_bootstrap.cc:654] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.846683 23960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53fd7f6fb694eb89eb611497e064ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.846972 23437 tablet_service.cc:1467] Processing CreateTablet for tablet a4f98ae45e914fa3981237af149b0666 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.847224 23437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4f98ae45e914fa3981237af149b0666. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.847247 23436 tablet_service.cc:1467] Processing CreateTablet for tablet f701e0b045d540698ba3879954d8b171 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.847492 23436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f701e0b045d540698ba3879954d8b171. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.848637 24465 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:13.849027 23435 tablet_service.cc:1467] Processing CreateTablet for tablet f53fd7f6fb694eb89eb611497e064ab5 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.849236 23435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53fd7f6fb694eb89eb611497e064ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.849382 24465 tablet_bootstrap.cc:654] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.850395 24465 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:13.850473 24465 ts_tablet_manager.cc:1367] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 19:56:13.850888 24465 raft_consensus.cc:348] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.850999 24465 raft_consensus.cc:374] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.851033 24465 raft_consensus.cc:729] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.851148 24465 consensus_queue.cc:260] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.851419 24465 ts_tablet_manager.cc:1398] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.851567 24465 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:13.852317 24465 tablet_bootstrap.cc:654] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.852525 23568 tablet_service.cc:1467] Processing CreateTablet for tablet f701e0b045d540698ba3879954d8b171 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.852525 23567 tablet_service.cc:1467] Processing CreateTablet for tablet f53fd7f6fb694eb89eb611497e064ab5 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.852752 23568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f701e0b045d540698ba3879954d8b171. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.852775 23698 tablet_service.cc:1467] Processing CreateTablet for tablet a4f98ae45e914fa3981237af149b0666 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.852917 23567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53fd7f6fb694eb89eb611497e064ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.853013 23698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4f98ae45e914fa3981237af149b0666. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.853339 24465 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:13.853423 24465 ts_tablet_manager.cc:1367] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:13.853791 24465 raft_consensus.cc:348] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.853902 24465 raft_consensus.cc:374] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.853940 24465 raft_consensus.cc:729] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.854063 24465 consensus_queue.cc:260] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.854318 24465 ts_tablet_manager.cc:1398] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.854465 24465 tablet_bootstrap.cc:492] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:13.854486 23699 tablet_service.cc:1467] Processing CreateTablet for tablet f701e0b045d540698ba3879954d8b171 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.854605 23305 tablet_service.cc:1467] Processing CreateTablet for tablet f53fd7f6fb694eb89eb611497e064ab5 (DEFAULT_TABLE table=rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:13.854707 23699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f701e0b045d540698ba3879954d8b171. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.854813 23305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53fd7f6fb694eb89eb611497e064ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:13.855367 24465 tablet_bootstrap.cc:654] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.856312 24465 tablet_bootstrap.cc:492] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a: No bootstrap required, opened a new log
I20240501 19:56:13.856405 24465 ts_tablet_manager.cc:1367] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:13.856532 24470 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700: Bootstrap starting.
I20240501 19:56:13.856782 24465 raft_consensus.cc:348] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.856875 24465 raft_consensus.cc:374] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.856895 24465 raft_consensus.cc:729] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.856981 24465 consensus_queue.cc:260] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.857183 24465 ts_tablet_manager.cc:1398] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:13.857273 24470 tablet_bootstrap.cc:654] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.857589 24466 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:13.857667 24466 ts_tablet_manager.cc:1367] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240501 19:56:13.858101 24466 raft_consensus.cc:348] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.858202 24466 raft_consensus.cc:374] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.858251 24466 raft_consensus.cc:729] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.858284 24470 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700: No bootstrap required, opened a new log
I20240501 19:56:13.858361 24470 ts_tablet_manager.cc:1367] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:13.858358 24466 consensus_queue.cc:260] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.858625 24466 ts_tablet_manager.cc:1398] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.858793 24466 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:13.858769 24470 raft_consensus.cc:348] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.858879 24470 raft_consensus.cc:374] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.858913 24470 raft_consensus.cc:729] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.859016 24470 consensus_queue.cc:260] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.859279 24470 ts_tablet_manager.cc:1398] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.859418 24470 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700: Bootstrap starting.
I20240501 19:56:13.859642 24466 tablet_bootstrap.cc:654] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.860159 24470 tablet_bootstrap.cc:654] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.860602 24464 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858: No bootstrap required, opened a new log
I20240501 19:56:13.860682 24464 ts_tablet_manager.cc:1367] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240501 19:56:13.861145 24464 raft_consensus.cc:348] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.861217 24470 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700: No bootstrap required, opened a new log
I20240501 19:56:13.861253 24464 raft_consensus.cc:374] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.861287 24464 raft_consensus.cc:729] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.861284 24470 ts_tablet_manager.cc:1367] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:13.861404 24464 consensus_queue.cc:260] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.861675 24464 ts_tablet_manager.cc:1398] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.861688 24470 raft_consensus.cc:348] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.861800 24470 raft_consensus.cc:374] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.861829 24470 raft_consensus.cc:729] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e47806645ae47278e35600cc67d5700, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.861835 24464 tablet_bootstrap.cc:492] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858: Bootstrap starting.
I20240501 19:56:13.861953 24470 consensus_queue.cc:260] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.862216 24470 ts_tablet_manager.cc:1398] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.862350 24520 raft_consensus.cc:482] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:13.862442 24520 raft_consensus.cc:504] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.862730 24464 tablet_bootstrap.cc:654] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.862897 24520 leader_election.cc:288] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:13.863168 23587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f98ae45e914fa3981237af149b0666" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:13.863229 23718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f98ae45e914fa3981237af149b0666" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700" is_pre_election: true
I20240501 19:56:13.863322 23718 raft_consensus.cc:2454] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 0.
W20240501 19:56:13.863600 23262 leader_election.cc:341] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:56:13.863706 23455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f98ae45e914fa3981237af149b0666" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a" is_pre_election: true
I20240501 19:56:13.863723 23848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f98ae45e914fa3981237af149b0666" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858" is_pre_election: true
I20240501 19:56:13.863857 23848 raft_consensus.cc:2454] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 0.
I20240501 19:56:13.863871 23455 raft_consensus.cc:2454] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 0.
I20240501 19:56:13.864115 23259 leader_election.cc:302] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 3 yes votes; 1 no votes. yes voters: 6e47806645ae47278e35600cc67d5700, ad9452840e4e4184a640fb63dab754fd, ee9bc706484e48fb8ff739a61cabe858; no voters: e8ddd74cfd954c7a8c2d284c335d7826
I20240501 19:56:13.864382 24520 raft_consensus.cc:2781] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:13.864449 24520 raft_consensus.cc:482] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:13.864491 24520 raft_consensus.cc:3037] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.864728 24464 tablet_bootstrap.cc:492] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858: No bootstrap required, opened a new log
I20240501 19:56:13.864804 24464 ts_tablet_manager.cc:1367] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:13.865208 24464 raft_consensus.cc:348] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.865317 24464 raft_consensus.cc:374] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.865356 24464 raft_consensus.cc:729] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.865391 24520 raft_consensus.cc:504] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.865465 24464 consensus_queue.cc:260] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.865721 24464 ts_tablet_manager.cc:1398] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.865849 24520 leader_election.cc:288] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 election: Requested vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657), ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:13.866225 23455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f98ae45e914fa3981237af149b0666" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a"
I20240501 19:56:13.866362 23455 raft_consensus.cc:3037] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.866371 23587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f98ae45e914fa3981237af149b0666" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
I20240501 19:56:13.866393 23718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f98ae45e914fa3981237af149b0666" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700"
I20240501 19:56:13.866469 23718 raft_consensus.cc:3037] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.867103 23718 raft_consensus.cc:2454] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 1.
I20240501 19:56:13.867218 23455 raft_consensus.cc:2454] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 1.
W20240501 19:56:13.867527 23262 leader_election.cc:341] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:56:13.868091 24467 tablet_bootstrap.cc:492] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1: Bootstrap starting.
I20240501 19:56:13.868703 24467 tablet_bootstrap.cc:654] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.869572 23848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4f98ae45e914fa3981237af149b0666" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858"
I20240501 19:56:13.869638 24467 tablet_bootstrap.cc:492] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1: No bootstrap required, opened a new log
I20240501 19:56:13.869696 23848 raft_consensus.cc:3037] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.869733 24467 ts_tablet_manager.cc:1367] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:13.870096 24467 raft_consensus.cc:348] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.870198 24467 raft_consensus.cc:374] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.870237 24467 raft_consensus.cc:729] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.870373 24467 consensus_queue.cc:260] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.870545 24468 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826: No bootstrap required, opened a new log
I20240501 19:56:13.870615 24467 ts_tablet_manager.cc:1398] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.870623 24468 ts_tablet_manager.cc:1367] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.001s
I20240501 19:56:13.870784 24467 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1: Bootstrap starting.
I20240501 19:56:13.871065 24468 raft_consensus.cc:348] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.871176 24468 raft_consensus.cc:374] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.871206 24468 raft_consensus.cc:729] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.871325 24468 consensus_queue.cc:260] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.871639 24468 ts_tablet_manager.cc:1398] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:13.872418 24468 tablet_bootstrap.cc:492] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:13.872553 23259 leader_election.cc:302] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 3 yes votes; 1 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 6e47806645ae47278e35600cc67d5700, ad9452840e4e4184a640fb63dab754fd; no voters: e8ddd74cfd954c7a8c2d284c335d7826
I20240501 19:56:13.872805 24520 raft_consensus.cc:2781] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:13.872893 24520 raft_consensus.cc:686] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Becoming Leader. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Running, Role: LEADER
I20240501 19:56:13.873015 24520 consensus_queue.cc:237] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } }
I20240501 19:56:13.873354 24468 tablet_bootstrap.cc:654] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.874581 24392 raft_consensus.cc:482] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:13.874681 24392 raft_consensus.cc:504] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.874987 24468 tablet_bootstrap.cc:492] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826: No bootstrap required, opened a new log
I20240501 19:56:13.875109 24468 ts_tablet_manager.cc:1367] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240501 19:56:13.875185 24392 leader_election.cc:288] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481)
I20240501 19:56:13.875478 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53fd7f6fb694eb89eb611497e064ab5" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:13.875603 23981 raft_consensus.cc:2454] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
I20240501 19:56:13.875784 24468 raft_consensus.cc:348] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.875866 23457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53fd7f6fb694eb89eb611497e064ab5" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a" is_pre_election: true
I20240501 19:56:13.875882 23587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53fd7f6fb694eb89eb611497e064ab5" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:13.875952 24468 raft_consensus.cc:374] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.875959 23457 raft_consensus.cc:2454] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
I20240501 19:56:13.875990 24468 raft_consensus.cc:729] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.876111 24468 consensus_queue.cc:260] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.876231 23783 leader_election.cc:302] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 4f51f982ad284c22aae01c4541a621f1, ee9bc706484e48fb8ff739a61cabe858; no voters:
I20240501 19:56:13.876358 23587 raft_consensus.cc:2454] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 0.
I20240501 19:56:13.876418 24468 ts_tablet_manager.cc:1398] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.876566 24468 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:13.876592 24431 raft_consensus.cc:2781] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:13.876642 24431 raft_consensus.cc:482] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:13.876682 24431 raft_consensus.cc:3037] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.877167 23184 catalog_manager.cc:5484] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd reported cstate change: term changed from 0 to 1, leader changed from <none> to ad9452840e4e4184a640fb63dab754fd (127.20.210.1). New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:13.877388 24468 tablet_bootstrap.cc:654] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.878055 24467 tablet_bootstrap.cc:654] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.878075 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53fd7f6fb694eb89eb611497e064ab5" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd" is_pre_election: true
I20240501 19:56:13.878754 23848 raft_consensus.cc:2454] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9452840e4e4184a640fb63dab754fd in term 1.
I20240501 19:56:13.878834 24431 raft_consensus.cc:504] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.879179 24467 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1: No bootstrap required, opened a new log
I20240501 19:56:13.879179 24468 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826: No bootstrap required, opened a new log
I20240501 19:56:13.879243 24468 ts_tablet_manager.cc:1367] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:13.879243 24467 ts_tablet_manager.cc:1367] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240501 19:56:13.879251 24431 leader_election.cc:288] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 election: Requested vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481)
I20240501 19:56:13.879652 24468 raft_consensus.cc:348] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.879637 24467 raft_consensus.cc:348] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.879745 24468 raft_consensus.cc:374] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.879745 24467 raft_consensus.cc:374] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.879774 24468 raft_consensus.cc:729] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.879777 24467 raft_consensus.cc:729] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f51f982ad284c22aae01c4541a621f1, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.879877 24468 consensus_queue.cc:260] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.879884 24467 consensus_queue.cc:260] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.880100 24468 ts_tablet_manager.cc:1398] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:13.880110 24467 ts_tablet_manager.cc:1398] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:13.880779 23587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53fd7f6fb694eb89eb611497e064ab5" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
I20240501 19:56:13.880887 23587 raft_consensus.cc:3037] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.881122 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53fd7f6fb694eb89eb611497e064ab5" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1"
I20240501 19:56:13.881201 23981 raft_consensus.cc:3037] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Advancing to term 1
W20240501 19:56:13.881196 23785 leader_election.cc:341] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:56:13.881618 23455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53fd7f6fb694eb89eb611497e064ab5" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a"
I20240501 19:56:13.881703 23455 raft_consensus.cc:3037] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.881705 23587 raft_consensus.cc:2454] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 1.
I20240501 19:56:13.882098 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53fd7f6fb694eb89eb611497e064ab5" candidate_uuid: "ee9bc706484e48fb8ff739a61cabe858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd"
W20240501 19:56:13.882493 23785 leader_election.cc:341] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:56:13.882526 23981 raft_consensus.cc:2454] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 1.
I20240501 19:56:13.882650 23455 raft_consensus.cc:2454] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee9bc706484e48fb8ff739a61cabe858 in term 1.
I20240501 19:56:13.883441 23783 leader_election.cc:302] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 3 yes votes; 1 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, e8ddd74cfd954c7a8c2d284c335d7826, ee9bc706484e48fb8ff739a61cabe858; no voters: ad9452840e4e4184a640fb63dab754fd
I20240501 19:56:13.883605 24431 raft_consensus.cc:2781] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:13.883627 24466 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:13.883704 24466 ts_tablet_manager.cc:1367] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.000s
I20240501 19:56:13.883694 24431 raft_consensus.cc:686] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Becoming Leader. State: Replica: ee9bc706484e48fb8ff739a61cabe858, State: Running, Role: LEADER
I20240501 19:56:13.883846 24431 consensus_queue.cc:237] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.884173 24466 raft_consensus.cc:348] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.884271 24466 raft_consensus.cc:374] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.884306 24466 raft_consensus.cc:729] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.884435 24466 consensus_queue.cc:260] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.884718 24466 ts_tablet_manager.cc:1398] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:13.884886 24466 tablet_bootstrap.cc:492] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd: Bootstrap starting.
I20240501 19:56:13.885753 24466 tablet_bootstrap.cc:654] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:13.886608 23184 catalog_manager.cc:5484] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5). New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:13.886755 24466 tablet_bootstrap.cc:492] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd: No bootstrap required, opened a new log
I20240501 19:56:13.886824 24466 ts_tablet_manager.cc:1367] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:13.887365 24466 raft_consensus.cc:348] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.887473 24466 raft_consensus.cc:374] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:13.887503 24466 raft_consensus.cc:729] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9452840e4e4184a640fb63dab754fd, State: Initialized, Role: FOLLOWER
I20240501 19:56:13.887600 24466 consensus_queue.cc:260] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:13.887842 24466 ts_tablet_manager.cc:1398] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:13.946398 24594 raft_consensus.cc:482] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:13.946540 24594 raft_consensus.cc:504] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.947146 24594 leader_election.cc:288] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657)
I20240501 19:56:13.947398 23326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f701e0b045d540698ba3879954d8b171" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd" is_pre_election: true
I20240501 19:56:13.947436 23587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f701e0b045d540698ba3879954d8b171" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:13.947436 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f701e0b045d540698ba3879954d8b171" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:13.947558 23587 raft_consensus.cc:2454] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 0.
I20240501 19:56:13.947558 23981 raft_consensus.cc:2454] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 0.
I20240501 19:56:13.947552 23326 raft_consensus.cc:2454] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 0.
I20240501 19:56:13.947777 23718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f701e0b045d540698ba3879954d8b171" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700" is_pre_election: true
I20240501 19:56:13.947906 23718 raft_consensus.cc:2454] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 0.
I20240501 19:56:13.948076 23391 leader_election.cc:302] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, 4f51f982ad284c22aae01c4541a621f1, ad9452840e4e4184a640fb63dab754fd; no voters:
I20240501 19:56:13.948246 24594 raft_consensus.cc:2781] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:13.948294 24594 raft_consensus.cc:482] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:13.948334 24594 raft_consensus.cc:3037] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.949142 24594 raft_consensus.cc:504] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.949568 24594 leader_election.cc:288] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 election: Requested vote from peers e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667), 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657)
I20240501 19:56:13.949923 23718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f701e0b045d540698ba3879954d8b171" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e47806645ae47278e35600cc67d5700"
I20240501 19:56:13.949963 23587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f701e0b045d540698ba3879954d8b171" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826"
I20240501 19:56:13.949963 23324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f701e0b045d540698ba3879954d8b171" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9452840e4e4184a640fb63dab754fd"
I20240501 19:56:13.950038 23718 raft_consensus.cc:3037] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.950063 23324 raft_consensus.cc:3037] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.950064 23587 raft_consensus.cc:3037] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.951001 23587 raft_consensus.cc:2454] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 1.
I20240501 19:56:13.951001 23324 raft_consensus.cc:2454] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 1.
I20240501 19:56:13.951297 23718 raft_consensus.cc:2454] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 1.
I20240501 19:56:13.951289 23392 leader_election.cc:302] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0496e037179e4b3ca72aead781c25d5a, ad9452840e4e4184a640fb63dab754fd, e8ddd74cfd954c7a8c2d284c335d7826; no voters:
I20240501 19:56:13.951485 24594 raft_consensus.cc:2781] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:13.951584 24594 raft_consensus.cc:686] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Becoming Leader. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Running, Role: LEADER
I20240501 19:56:13.951722 24594 consensus_queue.cc:237] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } }
I20240501 19:56:13.951995 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f701e0b045d540698ba3879954d8b171" candidate_uuid: "0496e037179e4b3ca72aead781c25d5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1"
I20240501 19:56:13.952116 23981 raft_consensus.cc:3037] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:13.953037 23981 raft_consensus.cc:2454] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0496e037179e4b3ca72aead781c25d5a in term 1.
I20240501 19:56:13.953668 23182 catalog_manager.cc:5484] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0496e037179e4b3ca72aead781c25d5a (127.20.210.2). New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } }
W20240501 19:56:14.073640 24633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:14.073835 24633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:14.076555 24633 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240501 19:56:14.162298 23718 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:14.162307 23981 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:14.162729 23587 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:14.163219 24311 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:14.163875 24592 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:14.164121 24592 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:14.164705 23324 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:14.165055 24594 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:14.167173 23848 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:14.168401 23719 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:14.168777 24395 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:14.170192 24520 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:14.175267 23455 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:14.175669 24519 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:14.175909 23586 raft_consensus.cc:3037] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:14.177018 23586 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:14.177552 24395 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:14.202596 24395 raft_consensus.cc:482] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:14.202711 24395 raft_consensus.cc:504] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } }
I20240501 19:56:14.203233 24395 leader_election.cc:288] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723), e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279), 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367), 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667)
I20240501 19:56:14.203928 23587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53fd7f6fb694eb89eb611497e064ab5" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" is_pre_election: true
I20240501 19:56:14.204078 23587 raft_consensus.cc:2379] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad9452840e4e4184a640fb63dab754fd in current term 1: Already voted for candidate ee9bc706484e48fb8ff739a61cabe858 in this term.
I20240501 19:56:14.210350 23850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53fd7f6fb694eb89eb611497e064ab5" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bc706484e48fb8ff739a61cabe858" is_pre_election: true
I20240501 19:56:14.211329 23981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53fd7f6fb694eb89eb611497e064ab5" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f51f982ad284c22aae01c4541a621f1" is_pre_election: true
I20240501 19:56:14.211477 23981 raft_consensus.cc:2379] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad9452840e4e4184a640fb63dab754fd in current term 1: Already voted for candidate ee9bc706484e48fb8ff739a61cabe858 in this term.
I20240501 19:56:14.211730 23260 leader_election.cc:302] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 5 voters: 1 yes votes; 3 no votes. yes voters: ad9452840e4e4184a640fb63dab754fd; no voters: 4f51f982ad284c22aae01c4541a621f1, e8ddd74cfd954c7a8c2d284c335d7826, ee9bc706484e48fb8ff739a61cabe858
I20240501 19:56:14.211885 24323 raft_consensus.cc:3037] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:14.212122 23457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53fd7f6fb694eb89eb611497e064ab5" candidate_uuid: "ad9452840e4e4184a640fb63dab754fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0496e037179e4b3ca72aead781c25d5a" is_pre_election: true
I20240501 19:56:14.212301 23457 raft_consensus.cc:2379] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ad9452840e4e4184a640fb63dab754fd in current term 1: Already voted for candidate ee9bc706484e48fb8ff739a61cabe858 in this term.
I20240501 19:56:14.212944 24323 raft_consensus.cc:2726] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 19:56:14.225726 23326 raft_consensus.cc:1261] T f53fd7f6fb694eb89eb611497e064ab5 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:14.226428 24630 consensus_queue.cc:1035] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:14.228539 23587 raft_consensus.cc:1261] T f53fd7f6fb694eb89eb611497e064ab5 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:14.228886 24630 consensus_queue.cc:1035] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:14.230685 23981 raft_consensus.cc:1261] T f53fd7f6fb694eb89eb611497e064ab5 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:14.231209 24630 consensus_queue.cc:1035] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:14.236260 23457 raft_consensus.cc:1261] T f53fd7f6fb694eb89eb611497e064ab5 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:14.236822 24431 consensus_queue.cc:1035] T f53fd7f6fb694eb89eb611497e064ab5 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:14.255016 24542 mvcc.cc:205] Tried to move back new op lower bound from 7022974460824543232 to 7022974459430567936. Current Snapshot: MvccSnapshot[applied={T|T < 7022974460824543232 or (T in {7022974460824543232})}]
INSERT report
rows total: 2000
time total: 151.714 ms
time per row: 0.0758571 ms
I20240501 19:56:14.264005 24319 mvcc.cc:205] Tried to move back new op lower bound from 7022974460620947456 to 7022974459385991168. Current Snapshot: MvccSnapshot[applied={T|T < 7022974460585013248}]
I20240501 19:56:14.278369 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.7:35019
--local_ip_for_outbound_sockets=127.20.210.7
--tserver_master_addrs=127.20.210.62:37935
--webserver_port=42705
--webserver_interface=127.20.210.7
--builtin_ntp_servers=127.20.210.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 19:56:14.389207 24675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:14.389395 24675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:14.389433 24675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:14.392017 24675 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:14.392068 24675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:14.392138 24675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.7
I20240501 19:56:14.395833 24675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.7:35019
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.20.210.7
--webserver_port=42705
--tserver_master_addrs=127.20.210.62:37935
--never_fsync=true
--heap_profile_path=/tmp/kudu.24675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.7
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:14.396476 24675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:14.397389 24675 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:14.405509 24681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:14.406188 24675 server_base.cc:1034] running on GCE node
W20240501 19:56:14.406791 24683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:14.407433 24680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:14.407733 24675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:14.408154 24675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:14.409337 24675 hybrid_clock.cc:648] HybridClock initialized: now 1714593374409322 us; error 25 us; skew 500 ppm
I20240501 19:56:14.410872 24675 webserver.cc:460] Webserver started at http://127.20.210.7:42705/ using document root <none> and password file <none>
I20240501 19:56:14.411338 24675 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:14.411410 24675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:14.414660 24675 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 19:56:14.416375 24689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:14.417340 24675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:14.417418 24675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "3b3f502cbeb1451087a06f95a3d93f2b"
format_stamp: "Formatted at 2024-05-01 19:56:07 on dist-test-slave-vgh4"
I20240501 19:56:14.417735 24675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:14.442728 24675 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:14.443267 24675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:14.443403 24675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:14.443866 24675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:14.444645 24675 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:14.444689 24675 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:14.444726 24675 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:14.444747 24675 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:14.452217 24675 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.7:35019
I20240501 19:56:14.452270 24802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.7:35019 every 8 connection(s)
I20240501 19:56:14.453126 24675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 19:56:14.461120 24803 heartbeater.cc:347] Connected to a master server at 127.20.210.62:37935
I20240501 19:56:14.461328 24803 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:14.461776 24803 heartbeater.cc:511] Master 127.20.210.62:37935 requested a full tablet report, sending...
I20240501 19:56:14.462811 23179 ts_manager.cc:194] Re-registered known tserver with Master: 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019)
I20240501 19:56:14.463299 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 24675
I20240501 19:56:14.463371 23179 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.7:46023
I20240501 19:56:14.463455 21320 test_util.cc:268] Using random seed: -519772706
I20240501 19:56:14.472680 21320 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_0 already exists
I20240501 19:56:14.472970 21320 test_util.cc:268] Using random seed: -519763194
I20240501 19:56:14.489519 21320 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_1 already exists
I20240501 19:56:14.489666 21320 test_util.cc:268] Using random seed: -519746495
I20240501 19:56:14.504009 21320 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_2 already exists
I20240501 19:56:14.504163 21320 test_util.cc:268] Using random seed: -519732000
I20240501 19:56:14.527674 21320 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_3 already exists
I20240501 19:56:14.527812 21320 test_util.cc:268] Using random seed: -519708349
I20240501 19:56:14.545495 21320 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_4 already exists
W20240501 19:56:15.180379 23652 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.5:44723, user_credentials={real_user=slave}} blocked reactor thread for 49741us
I20240501 19:56:15.463322 23568 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:15.464977 23961 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:15.465274 24737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:15.465304 24803 heartbeater.cc:503] Master 127.20.210.62:37935 was elected leader, sending a full tablet report...
I20240501 19:56:15.472429 23435 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:15.564873 23698 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:15.566324 23305 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:15.684633 23830 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
W20240501 19:56:15.928478 23786 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.3:37279, user_credentials={real_user=slave}} blocked reactor thread for 50674.1us
W20240501 19:56:16.212582 23786 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.3:37279, user_credentials={real_user=slave}} blocked reactor thread for 45906us
I20240501 19:56:16.740515 23435 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:16.792976 23305 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:16.800582 23830 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:16.802335 23568 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:16.826367 23961 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:16.844784 24737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:16.859892 23698 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:17.291599 23325 consensus_queue.cc:237] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 53, Committed index: 53, Last appended: 1.54, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:17.295585 23588 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 54. Preceding OpId from leader: term: 1 index: 55. (index mismatch)
I20240501 19:56:17.295585 23456 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 53. Preceding OpId from leader: term: 1 index: 55. (index mismatch)
I20240501 19:56:17.295874 24519 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 53, Time since last communication: 0.000s
I20240501 19:56:17.296970 23846 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 54. Preceding OpId from leader: term: 1 index: 55. (index mismatch)
I20240501 19:56:17.297075 24519 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 53, Time since last communication: 0.000s
I20240501 19:56:17.298596 24519 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 53, Time since last communication: 0.000s
I20240501 19:56:17.301322 23976 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 53. Preceding OpId from leader: term: 1 index: 55. (index mismatch)
I20240501 19:56:17.301750 24519 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 53, Time since last communication: 0.000s
I20240501 19:56:17.304811 25022 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.55: config changed from index -1 to 55, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:17.311988 23179 catalog_manager.cc:5484] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index -1 to 55, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:17.317166 23583 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.55: config changed from index -1 to 55, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:17.320770 23456 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.55: config changed from index -1 to 55, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:17.359814 23849 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.55: config changed from index -1 to 55, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:17.396984 23976 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.55: config changed from index -1 to 55, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
W20240501 19:56:17.412767 23260 consensus_peers.cc:485] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd -> Peer 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019): Couldn't send request to peer 3b3f502cbeb1451087a06f95a3d93f2b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c9623f516f84445ade4bf06eb05673f. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:17.431939 23588 consensus_queue.cc:237] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 48, Committed index: 48, Last appended: 1.48, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: true } }
I20240501 19:56:17.433892 23976 raft_consensus.cc:1261] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer e8ddd74cfd954c7a8c2d284c335d7826: Log matching property violated. Preceding OpId in replica: term: 1 index: 48. Preceding OpId from leader: term: 1 index: 49. (index mismatch)
I20240501 19:56:17.434185 23716 raft_consensus.cc:1261] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer e8ddd74cfd954c7a8c2d284c335d7826: Log matching property violated. Preceding OpId in replica: term: 1 index: 48. Preceding OpId from leader: term: 1 index: 49. (index mismatch)
I20240501 19:56:17.434365 25047 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20240501 19:56:17.434865 25047 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
W20240501 19:56:17.435518 23521 consensus_peers.cc:485] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 -> Peer 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367): Couldn't send request to peer 0496e037179e4b3ca72aead781c25d5a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: af8e2ce72fb24a18b54b0d2e49383b41. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:17.436210 23848 raft_consensus.cc:1261] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer e8ddd74cfd954c7a8c2d284c335d7826: Log matching property violated. Preceding OpId in replica: term: 1 index: 48. Preceding OpId from leader: term: 1 index: 49. (index mismatch)
I20240501 19:56:17.436599 25047 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20240501 19:56:17.439404 24969 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 LEADER]: Committing config change with OpId 1.49: config changed from index -1 to 49, NON_VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) added. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: true } } }
I20240501 19:56:17.439802 23719 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.49: config changed from index -1 to 49, NON_VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) added. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: true } } }
I20240501 19:56:17.441558 23980 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.49: config changed from index -1 to 49, NON_VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) added. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: true } } }
I20240501 19:56:17.443938 23848 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.49: config changed from index -1 to 49, NON_VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) added. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: true } } }
I20240501 19:56:17.444747 23177 catalog_manager.cc:5484] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 reported cstate change: config changed from index -1 to 49, NON_VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) added. New cstate: current_term: 1 leader_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" committed_config { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: true } } }
I20240501 19:56:17.446635 23322 raft_consensus.cc:1261] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer e8ddd74cfd954c7a8c2d284c335d7826: Log matching property violated. Preceding OpId in replica: term: 1 index: 48. Preceding OpId from leader: term: 1 index: 49. (index mismatch)
I20240501 19:56:17.447263 24882 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20240501 19:56:17.448482 23322 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.49: config changed from index -1 to 49, NON_VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) added. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: true } } }
I20240501 19:56:17.588165 23454 consensus_queue.cc:237] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 1.43, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:17.589807 23321 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20240501 19:56:17.590173 24927 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 42, Time since last communication: 0.000s
I20240501 19:56:17.597312 23978 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20240501 19:56:17.610277 23718 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
I20240501 19:56:17.610594 23588 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
W20240501 19:56:17.610178 23391 consensus_peers.cc:485] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a -> Peer 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019): Couldn't send request to peer 3b3f502cbeb1451087a06f95a3d93f2b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f701e0b045d540698ba3879954d8b171. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:17.611274 24462 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 42, Time since last communication: 0.000s
I20240501 19:56:17.611466 24927 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 42, Time since last communication: 0.000s
I20240501 19:56:17.613306 24462 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:17.614395 23718 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:17.617086 23588 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:17.618968 23182 catalog_manager.cc:5484] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a reported cstate change: config changed from index -1 to 44, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:17.639097 24926 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 42, Time since last communication: 0.000s
I20240501 19:56:17.640473 23323 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:17.642939 23979 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index -1 to 44, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:17.739023 23719 consensus_queue.cc:237] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 60, Committed index: 60, Last appended: 1.60, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: true } }
W20240501 19:56:17.740648 23655 consensus_peers.cc:485] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 -> Peer e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279): Couldn't send request to peer e8ddd74cfd954c7a8c2d284c335d7826. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f549fed054d433381ac6d38d64c74bb. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:17.740732 23322 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 19:56:17.741056 23457 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 19:56:17.741113 24895 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20240501 19:56:17.741420 24895 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20240501 19:56:17.743966 25073 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.61: config changed from index -1 to 61, NON_VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) added. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: true } } }
I20240501 19:56:17.747327 23980 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 19:56:17.752012 23457 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index -1 to 61, NON_VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) added. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: true } } }
I20240501 19:56:17.753407 23184 catalog_manager.cc:5484] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 reported cstate change: config changed from index -1 to 61, NON_VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) added. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:17.760435 24971 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20240501 19:56:17.763221 23980 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index -1 to 61, NON_VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) added. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: true } } }
I20240501 19:56:17.769621 23322 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index -1 to 61, NON_VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) added. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: true } } }
I20240501 19:56:17.809217 23850 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 19:56:17.819209 24540 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20240501 19:56:17.826078 23846 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index -1 to 61, NON_VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) added. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: true } } }
I20240501 19:56:17.919098 25089 ts_tablet_manager.cc:902] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b: Initiating tablet copy from peer ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481)
I20240501 19:56:17.920837 25089 tablet_copy_client.cc:308] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:35481
I20240501 19:56:17.962235 23325 consensus_queue.cc:237] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 65, Committed index: 65, Last appended: 1.65, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
W20240501 19:56:17.964768 23260 consensus_peers.cc:485] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd -> Peer 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019): Couldn't send request to peer 3b3f502cbeb1451087a06f95a3d93f2b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b1a0cebcdad24e34abc11eef9d4d891d. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:17.965376 23719 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240501 19:56:17.965709 24395 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240501 19:56:17.970067 23977 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240501 19:56:17.970394 23848 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240501 19:56:17.974994 24395 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240501 19:56:17.980600 25022 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:17.983461 23182 catalog_manager.cc:5484] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index -1 to 66, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:17.986526 24519 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240501 19:56:17.987396 23457 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240501 19:56:17.987496 23977 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.003408 23719 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.016649 24902 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 64, Time since last communication: 0.000s
I20240501 19:56:18.018752 23848 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.029613 23457 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.039124 23455 consensus_queue.cc:237] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 73, Committed index: 73, Last appended: 1.73, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:18.047616 23848 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
W20240501 19:56:18.048014 23391 consensus_peers.cc:485] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a -> Peer 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019): Couldn't send request to peer 3b3f502cbeb1451087a06f95a3d93f2b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a4ef2fdbb40e48b18025136adb39defc. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:18.048447 24594 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 19:56:18.059818 23978 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240501 19:56:18.062220 23588 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240501 19:56:18.062994 23718 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240501 19:56:18.065495 24926 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 19:56:18.070523 24594 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Committing config change with OpId 1.74: config changed from index -1 to 74, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.075083 23848 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index -1 to 74, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.076047 23182 catalog_manager.cc:5484] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a reported cstate change: config changed from index -1 to 74, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:18.085696 23978 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index -1 to 74, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.087164 23346 tablet_copy_service.cc:140] P ad9452840e4e4184a640fb63dab754fd: Received BeginTabletCopySession request for tablet 1c9623f516f84445ade4bf06eb05673f from peer 3b3f502cbeb1451087a06f95a3d93f2b ({username='slave'} at 127.20.210.7:35207)
I20240501 19:56:18.087298 23346 tablet_copy_service.cc:161] P ad9452840e4e4184a640fb63dab754fd: Beginning new tablet copy session on tablet 1c9623f516f84445ade4bf06eb05673f from peer 3b3f502cbeb1451087a06f95a3d93f2b at {username='slave'} at 127.20.210.7:35207: session id = 3b3f502cbeb1451087a06f95a3d93f2b-1c9623f516f84445ade4bf06eb05673f
I20240501 19:56:18.088603 23346 tablet_copy_source_session.cc:205] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:18.094882 25113 ts_tablet_manager.cc:902] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: Initiating tablet copy from peer 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:18.095450 25113 tablet_copy_client.cc:308] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:38367
I20240501 19:56:18.096127 25089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c9623f516f84445ade4bf06eb05673f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:18.100174 23477 tablet_copy_service.cc:140] P 0496e037179e4b3ca72aead781c25d5a: Received BeginTabletCopySession request for tablet f701e0b045d540698ba3879954d8b171 from peer 3b3f502cbeb1451087a06f95a3d93f2b ({username='slave'} at 127.20.210.7:36339)
I20240501 19:56:18.100302 23477 tablet_copy_service.cc:161] P 0496e037179e4b3ca72aead781c25d5a: Beginning new tablet copy session on tablet f701e0b045d540698ba3879954d8b171 from peer 3b3f502cbeb1451087a06f95a3d93f2b at {username='slave'} at 127.20.210.7:36339: session id = 3b3f502cbeb1451087a06f95a3d93f2b-f701e0b045d540698ba3879954d8b171
I20240501 19:56:18.101639 23477 tablet_copy_source_session.cc:205] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:18.102860 25113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f701e0b045d540698ba3879954d8b171. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:18.117043 24988 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 19:56:18.120332 25042 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 19:56:18.123812 23587 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index -1 to 74, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.134464 23719 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index -1 to 74, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.141636 25107 ts_tablet_manager.cc:902] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a: Initiating tablet copy from peer e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279)
I20240501 19:56:18.143743 25113 tablet_copy_client.cc:783] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:18.143862 25089 tablet_copy_client.cc:783] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:18.143967 25113 tablet_copy_client.cc:647] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:18.144029 25089 tablet_copy_client.cc:647] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:18.148571 25089 tablet_copy_client.cc:523] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:18.149780 25113 tablet_copy_client.cc:523] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:18.151665 25113 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap starting.
I20240501 19:56:18.174968 25113 log.cc:824] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:18.188513 25089 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap starting.
I20240501 19:56:18.283684 25107 tablet_copy_client.cc:308] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:37279
I20240501 19:56:18.286437 25124 ts_tablet_manager.cc:902] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826: Initiating tablet copy from peer 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657)
I20240501 19:56:18.287367 25124 tablet_copy_client.cc:308] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:44657
I20240501 19:56:18.289026 23739 tablet_copy_service.cc:140] P 6e47806645ae47278e35600cc67d5700: Received BeginTabletCopySession request for tablet 6f549fed054d433381ac6d38d64c74bb from peer e8ddd74cfd954c7a8c2d284c335d7826 ({username='slave'} at 127.20.210.3:59519)
I20240501 19:56:18.289170 23739 tablet_copy_service.cc:161] P 6e47806645ae47278e35600cc67d5700: Beginning new tablet copy session on tablet 6f549fed054d433381ac6d38d64c74bb from peer e8ddd74cfd954c7a8c2d284c335d7826 at {username='slave'} at 127.20.210.3:59519: session id = e8ddd74cfd954c7a8c2d284c335d7826-6f549fed054d433381ac6d38d64c74bb
I20240501 19:56:18.290840 23739 tablet_copy_source_session.cc:205] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:18.293889 25124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f549fed054d433381ac6d38d64c74bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:18.299438 25124 tablet_copy_client.cc:783] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:18.299660 25124 tablet_copy_client.cc:647] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:18.313095 25089 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=72 overwritten=0 applied=72 ignored=0} inserts{seen=744 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:18.317312 23608 tablet_copy_service.cc:140] P e8ddd74cfd954c7a8c2d284c335d7826: Received BeginTabletCopySession request for tablet af8e2ce72fb24a18b54b0d2e49383b41 from peer 0496e037179e4b3ca72aead781c25d5a ({username='slave'} at 127.20.210.2:60845)
I20240501 19:56:18.317441 23608 tablet_copy_service.cc:161] P e8ddd74cfd954c7a8c2d284c335d7826: Beginning new tablet copy session on tablet af8e2ce72fb24a18b54b0d2e49383b41 from peer 0496e037179e4b3ca72aead781c25d5a at {username='slave'} at 127.20.210.2:60845: session id = 0496e037179e4b3ca72aead781c25d5a-af8e2ce72fb24a18b54b0d2e49383b41
I20240501 19:56:18.336403 23717 consensus_queue.cc:237] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 60, Committed index: 60, Last appended: 1.60, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:18.338399 23322 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 19:56:18.338524 23582 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 19:56:18.338734 24971 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20240501 19:56:18.338886 24849 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20240501 19:56:18.339020 23976 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 19:56:18.339778 24848 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
W20240501 19:56:18.349249 23653 consensus_peers.cc:485] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 -> Peer 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019): Couldn't send request to peer 3b3f502cbeb1451087a06f95a3d93f2b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9f0625a730b0408b963d94f24e25fed7. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:18.355006 25089 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap complete.
I20240501 19:56:18.355124 23452 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 19:56:18.355379 25089 ts_tablet_manager.cc:1367] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent bootstrapping tablet: real 0.167s user 0.072s sys 0.005s
W20240501 19:56:18.355747 24868 log.cc:924] Time spent T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20240501 19:56:18.356034 23608 tablet_copy_source_session.cc:205] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:18.357923 25107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af8e2ce72fb24a18b54b0d2e49383b41. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:18.360172 25107 tablet_copy_client.cc:783] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a: tablet copy: Starting download of 0 data blocks...
W20240501 19:56:18.360240 24854 log.cc:924] Time spent T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
I20240501 19:56:18.360314 25107 tablet_copy_client.cc:647] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a: tablet copy: Starting download of 1 WAL segments...
W20240501 19:56:18.360556 24847 log.cc:924] Time spent T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
I20240501 19:56:18.360754 24848 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 59, Time since last communication: 0.000s
I20240501 19:56:18.360989 25089 raft_consensus.cc:348] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:18.361238 25089 raft_consensus.cc:729] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3b3f502cbeb1451087a06f95a3d93f2b, State: Initialized, Role: LEARNER
I20240501 19:56:18.361593 25089 consensus_queue.cc:260] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 72, Last appended: 1.72, Last appended by leader: 72, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 55 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
W20240501 19:56:18.362156 24869 log.cc:924] Time spent T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20240501 19:56:18.362490 25089 ts_tablet_manager.cc:1398] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20240501 19:56:18.363454 25124 tablet_copy_client.cc:523] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:18.363749 24971 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.61: config changed from index -1 to 61, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.365074 23346 tablet_copy_service.cc:342] P ad9452840e4e4184a640fb63dab754fd: Request end of tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-1c9623f516f84445ade4bf06eb05673f received from {username='slave'} at 127.20.210.7:35207
I20240501 19:56:18.365154 23346 tablet_copy_service.cc:433] P ad9452840e4e4184a640fb63dab754fd: ending tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-1c9623f516f84445ade4bf06eb05673f on tablet 1c9623f516f84445ade4bf06eb05673f with peer 3b3f502cbeb1451087a06f95a3d93f2b
I20240501 19:56:18.365280 25124 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap starting.
I20240501 19:56:18.366973 23184 catalog_manager.cc:5484] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 reported cstate change: config changed from index -1 to 61, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:18.375769 23584 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index -1 to 61, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.386634 23976 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index -1 to 61, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.390398 23323 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index -1 to 61, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.425488 25089 ts_tablet_manager.cc:902] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b: Initiating tablet copy from peer ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481)
I20240501 19:56:18.425971 25089 tablet_copy_client.cc:308] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:35481
I20240501 19:56:18.426764 23346 tablet_copy_service.cc:140] P ad9452840e4e4184a640fb63dab754fd: Received BeginTabletCopySession request for tablet b1a0cebcdad24e34abc11eef9d4d891d from peer 3b3f502cbeb1451087a06f95a3d93f2b ({username='slave'} at 127.20.210.7:35207)
I20240501 19:56:18.426887 23346 tablet_copy_service.cc:161] P ad9452840e4e4184a640fb63dab754fd: Beginning new tablet copy session on tablet b1a0cebcdad24e34abc11eef9d4d891d from peer 3b3f502cbeb1451087a06f95a3d93f2b at {username='slave'} at 127.20.210.7:35207: session id = 3b3f502cbeb1451087a06f95a3d93f2b-b1a0cebcdad24e34abc11eef9d4d891d
I20240501 19:56:18.427739 23346 tablet_copy_source_session.cc:205] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:18.429881 23457 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index -1 to 61, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:18.430218 25113 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=53 overwritten=0 applied=53 ignored=0} inserts{seen=684 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:18.430526 25113 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap complete.
I20240501 19:56:18.430701 25113 ts_tablet_manager.cc:1367] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent bootstrapping tablet: real 0.279s user 0.058s sys 0.009s
I20240501 19:56:18.431257 25113 raft_consensus.cc:348] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:18.431380 25113 raft_consensus.cc:729] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3b3f502cbeb1451087a06f95a3d93f2b, State: Initialized, Role: LEARNER
I20240501 19:56:18.431476 25113 consensus_queue.cc:260] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 53, Last appended: 1.53, Last appended by leader: 53, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:18.432098 25089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1a0cebcdad24e34abc11eef9d4d891d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:18.432117 25113 ts_tablet_manager.cc:1398] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:18.434468 23477 tablet_copy_service.cc:342] P 0496e037179e4b3ca72aead781c25d5a: Request end of tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-f701e0b045d540698ba3879954d8b171 received from {username='slave'} at 127.20.210.7:36339
I20240501 19:56:18.434540 23477 tablet_copy_service.cc:433] P 0496e037179e4b3ca72aead781c25d5a: ending tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-f701e0b045d540698ba3879954d8b171 on tablet f701e0b045d540698ba3879954d8b171 with peer 3b3f502cbeb1451087a06f95a3d93f2b
I20240501 19:56:18.447675 25107 tablet_copy_client.cc:523] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:18.449084 25113 ts_tablet_manager.cc:902] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b: Initiating tablet copy from peer 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:18.449330 25107 tablet_bootstrap.cc:492] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a: Bootstrap starting.
I20240501 19:56:18.449930 25113 tablet_copy_client.cc:308] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:38367
I20240501 19:56:18.452263 23477 tablet_copy_service.cc:140] P 0496e037179e4b3ca72aead781c25d5a: Received BeginTabletCopySession request for tablet a4ef2fdbb40e48b18025136adb39defc from peer 3b3f502cbeb1451087a06f95a3d93f2b ({username='slave'} at 127.20.210.7:36339)
I20240501 19:56:18.452353 23477 tablet_copy_service.cc:161] P 0496e037179e4b3ca72aead781c25d5a: Beginning new tablet copy session on tablet a4ef2fdbb40e48b18025136adb39defc from peer 3b3f502cbeb1451087a06f95a3d93f2b at {username='slave'} at 127.20.210.7:36339: session id = 3b3f502cbeb1451087a06f95a3d93f2b-a4ef2fdbb40e48b18025136adb39defc
I20240501 19:56:18.453269 23477 tablet_copy_source_session.cc:205] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:18.458274 25089 tablet_copy_client.cc:783] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:18.458483 25089 tablet_copy_client.cc:647] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:18.465490 25113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ef2fdbb40e48b18025136adb39defc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:18.467576 25113 tablet_copy_client.cc:783] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:18.467722 25113 tablet_copy_client.cc:647] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:18.469130 25113 tablet_copy_client.cc:523] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:18.470732 25113 tablet_bootstrap.cc:492] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap starting.
I20240501 19:56:18.519068 25089 tablet_copy_client.cc:523] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:18.520679 25089 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap starting.
I20240501 19:56:18.532008 24757 raft_consensus.cc:1203] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.43->[1.44-1.57] Dedup: 1.53->[1.54-1.57]
I20240501 19:56:18.574676 25042 raft_consensus.cc:1050] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a: attempting to promote NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b to VOTER
I20240501 19:56:18.575189 25042 consensus_queue.cc:237] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 57, Committed index: 57, Last appended: 1.58, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:18.577862 23584 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 57. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 19:56:18.577970 23716 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 19:56:18.578347 24988 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 57, Time since last communication: 0.000s
I20240501 19:56:18.578608 24757 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 19:56:18.578665 24988 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 57, Time since last communication: 0.000s
I20240501 19:56:18.579238 24926 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 57, Time since last communication: 0.000s
I20240501 19:56:18.580689 23321 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 19:56:18.581369 24926 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 57, Time since last communication: 0.000s
I20240501 19:56:18.582707 23586 raft_consensus.cc:1203] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.57->[1.58-1.58] Dedup: 1.58->[]
I20240501 19:56:18.584416 24926 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Committing config change with OpId 1.59: config changed from index 44 to 59, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.585546 23979 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 58. Preceding OpId from leader: term: 1 index: 59. (index mismatch)
I20240501 19:56:18.586836 23325 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 44 to 59, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.588984 23184 catalog_manager.cc:5484] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a reported cstate change: config changed from index 44 to 59, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:18.591297 23586 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 44 to 59, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.594154 24926 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 59, Last known committed idx: 57, Time since last communication: 0.000s
I20240501 19:56:18.595175 23718 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 44 to 59, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.604060 24757 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 44 to 59, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.605581 23978 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.59: config changed from index 44 to 59, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 59 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.625181 23454 consensus_queue.cc:237] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 60, Committed index: 60, Last appended: 1.60, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:18.627920 24757 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 19:56:18.629384 23587 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 19:56:18.629886 24926 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 59, Time since last communication: 0.000s
I20240501 19:56:18.632066 24594 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20240501 19:56:18.647809 24462 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Committing config change with OpId 1.61: config changed from index 59 to 61, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.650173 23168 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f701e0b045d540698ba3879954d8b171 with cas_config_opid_index 59: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:18.651104 23587 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index 59 to 61, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.656988 23718 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 19:56:18.650923 24757 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index 59 to 61, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.661886 23182 catalog_manager.cc:5484] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 reported cstate change: config changed from index 59 to 61, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.668452 23976 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 60. Preceding OpId from leader: term: 1 index: 61. (index mismatch)
I20240501 19:56:18.669692 23305 tablet_service.cc:1514] Processing DeleteTablet for tablet f701e0b045d540698ba3879954d8b171 with delete_type TABLET_DATA_TOMBSTONED (TS ad9452840e4e4184a640fb63dab754fd not found in new config with opid_index 61) from {username='slave'} at 127.0.0.1:55042
I20240501 19:56:18.674083 24594 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20240501 19:56:18.674348 24594 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 61, Last known committed idx: 60, Time since last communication: 0.000s
I20240501 19:56:18.678248 23718 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index 59 to 61, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.679889 23980 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.61: config changed from index 59 to 61, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.688897 25171 tablet_replica.cc:330] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd: stopping tablet replica
I20240501 19:56:18.690601 25171 raft_consensus.cc:2229] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:18.691026 25171 raft_consensus.cc:2258] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:18.697814 25171 ts_tablet_manager.cc:1875] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:18.701012 25171 ts_tablet_manager.cc:1888] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.60
I20240501 19:56:18.701149 25171 log.cc:1196] T f701e0b045d540698ba3879954d8b171 P ad9452840e4e4184a640fb63dab754fd: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/f701e0b045d540698ba3879954d8b171
I20240501 19:56:18.702843 23170 catalog_manager.cc:4830] TS ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481): tablet f701e0b045d540698ba3879954d8b171 (table rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]) successfully deleted
I20240501 19:56:18.712255 24756 raft_consensus.cc:1203] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.54->[1.55-1.78] Dedup: 1.72->[1.73-1.78]
I20240501 19:56:18.740149 25022 raft_consensus.cc:1050] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd: attempting to promote NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b to VOTER
I20240501 19:56:18.740615 25022 consensus_queue.cc:237] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 78, Committed index: 78, Last appended: 1.79, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:18.742564 24756 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 79. Preceding OpId from leader: term: 1 index: 80. (index mismatch)
I20240501 19:56:18.743260 23848 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 80. (index mismatch)
I20240501 19:56:18.743520 24395 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 80, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 19:56:18.744251 23979 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 80. (index mismatch)
I20240501 19:56:18.744799 24519 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 80, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 19:56:18.752499 23980 raft_consensus.cc:1203] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.78->[1.79-1.79] Dedup: 1.79->[]
I20240501 19:56:18.754397 24519 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 80, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 19:56:18.756533 23584 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 79. Preceding OpId from leader: term: 1 index: 80. (index mismatch)
I20240501 19:56:18.756906 24519 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 80, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 19:56:18.770395 23455 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 79. Preceding OpId from leader: term: 1 index: 80. (index mismatch)
I20240501 19:56:18.774098 24519 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.80: config changed from index 55 to 80, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.777632 24756 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.80: config changed from index 55 to 80, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.785637 23184 catalog_manager.cc:5484] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index 55 to 80, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:18.785897 23849 raft_consensus.cc:1203] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.78->[1.79-1.80] Dedup: 1.79->[1.80-1.80]
I20240501 19:56:18.788193 23586 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.80: config changed from index 55 to 80, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.793751 24902 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 80, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 19:56:18.819533 23976 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.80: config changed from index 55 to 80, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.828984 23455 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.80: config changed from index 55 to 80, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.837512 23321 consensus_queue.cc:237] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 81, Committed index: 81, Last appended: 1.81, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:18.839152 24756 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 81. Preceding OpId from leader: term: 1 index: 82. (index mismatch)
I20240501 19:56:18.839617 24519 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 82, Last known committed idx: 81, Time since last communication: 0.000s
I20240501 19:56:18.841383 23849 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.80: config changed from index 55 to 80, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.844182 23588 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 81. Preceding OpId from leader: term: 1 index: 82. (index mismatch)
I20240501 19:56:18.844604 24519 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 82, Last known committed idx: 81, Time since last communication: 0.000s
I20240501 19:56:18.846068 23981 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 81. Preceding OpId from leader: term: 1 index: 82. (index mismatch)
I20240501 19:56:18.847427 25124 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap replayed 1/1 log segments. Stats: ops{read=63 overwritten=0 applied=63 ignored=0} inserts{seen=735 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:18.847409 24519 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 82, Last known committed idx: 81, Time since last communication: 0.000s
I20240501 19:56:18.847642 24519 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.82: config changed from index 80 to 82, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.847800 25124 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826: Bootstrap complete.
I20240501 19:56:18.847779 23586 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.82: config changed from index 80 to 82, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.848018 25124 ts_tablet_manager.cc:1367] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826: Time spent bootstrapping tablet: real 0.483s user 0.062s sys 0.003s
I20240501 19:56:18.848372 25124 raft_consensus.cc:348] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: true } }
I20240501 19:56:18.848453 25124 raft_consensus.cc:729] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e8ddd74cfd954c7a8c2d284c335d7826, State: Initialized, Role: LEARNER
I20240501 19:56:18.848528 25124 consensus_queue.cc:260] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 63, Last appended: 1.63, Last appended by leader: 63, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: true } }
I20240501 19:56:18.849232 23170 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1c9623f516f84445ade4bf06eb05673f with cas_config_opid_index 80: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:18.851539 23981 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.82: config changed from index 80 to 82, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.856158 25124 ts_tablet_manager.cc:1398] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240501 19:56:18.856981 23739 tablet_copy_service.cc:342] P 6e47806645ae47278e35600cc67d5700: Request end of tablet copy session e8ddd74cfd954c7a8c2d284c335d7826-6f549fed054d433381ac6d38d64c74bb received from {username='slave'} at 127.20.210.3:59519
I20240501 19:56:18.857066 23739 tablet_copy_service.cc:433] P 6e47806645ae47278e35600cc67d5700: ending tablet copy session e8ddd74cfd954c7a8c2d284c335d7826-6f549fed054d433381ac6d38d64c74bb on tablet 6f549fed054d433381ac6d38d64c74bb with peer e8ddd74cfd954c7a8c2d284c335d7826
I20240501 19:56:18.859882 24756 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.82: config changed from index 80 to 82, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.857736 23184 catalog_manager.cc:5484] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 reported cstate change: config changed from index 80 to 82, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:18.873535 23435 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c9623f516f84445ade4bf06eb05673f with delete_type TABLET_DATA_TOMBSTONED (TS 0496e037179e4b3ca72aead781c25d5a not found in new config with opid_index 82) from {username='slave'} at 127.0.0.1:49870
I20240501 19:56:18.889797 25185 tablet_replica.cc:330] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a: stopping tablet replica
I20240501 19:56:18.890064 25185 raft_consensus.cc:2229] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:18.890352 25185 raft_consensus.cc:2258] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:18.892374 25146 ts_tablet_manager.cc:902] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b: Initiating tablet copy from peer 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657)
I20240501 19:56:18.892843 25146 tablet_copy_client.cc:308] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:44657
I20240501 19:56:18.894948 23849 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 81. Preceding OpId from leader: term: 1 index: 82. (index mismatch)
I20240501 19:56:18.896212 25185 ts_tablet_manager.cc:1875] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:18.908983 25185 ts_tablet_manager.cc:1888] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.80
I20240501 19:56:18.909071 25185 log.cc:1196] T 1c9623f516f84445ade4bf06eb05673f P 0496e037179e4b3ca72aead781c25d5a: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/1c9623f516f84445ade4bf06eb05673f
I20240501 19:56:18.911780 23168 catalog_manager.cc:4830] TS 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367): tablet 1c9623f516f84445ade4bf06eb05673f (table rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]) successfully deleted
I20240501 19:56:18.921381 23739 tablet_copy_service.cc:140] P 6e47806645ae47278e35600cc67d5700: Received BeginTabletCopySession request for tablet 9f0625a730b0408b963d94f24e25fed7 from peer 3b3f502cbeb1451087a06f95a3d93f2b ({username='slave'} at 127.20.210.7:54081)
I20240501 19:56:18.921468 23739 tablet_copy_service.cc:161] P 6e47806645ae47278e35600cc67d5700: Beginning new tablet copy session on tablet 9f0625a730b0408b963d94f24e25fed7 from peer 3b3f502cbeb1451087a06f95a3d93f2b at {username='slave'} at 127.20.210.7:54081: session id = 3b3f502cbeb1451087a06f95a3d93f2b-9f0625a730b0408b963d94f24e25fed7
I20240501 19:56:18.922374 23739 tablet_copy_source_session.cc:205] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:18.927685 25146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f0625a730b0408b963d94f24e25fed7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:18.929630 25146 tablet_copy_client.cc:783] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:18.929754 25146 tablet_copy_client.cc:647] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:18.934907 25146 tablet_copy_client.cc:523] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:18.936229 25146 tablet_bootstrap.cc:492] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap starting.
I20240501 19:56:18.939882 25089 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=66 overwritten=0 applied=66 ignored=0} inserts{seen=751 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:18.940119 25089 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap complete.
I20240501 19:56:18.940297 25089 ts_tablet_manager.cc:1367] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent bootstrapping tablet: real 0.420s user 0.058s sys 0.008s
I20240501 19:56:18.940665 25089 raft_consensus.cc:348] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:18.940778 25089 raft_consensus.cc:729] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3b3f502cbeb1451087a06f95a3d93f2b, State: Initialized, Role: LEARNER
I20240501 19:56:18.940868 25089 consensus_queue.cc:260] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 66, Last appended: 1.66, Last appended by leader: 66, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:18.941113 25089 ts_tablet_manager.cc:1398] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:18.941426 25113 tablet_bootstrap.cc:492] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=79 overwritten=0 applied=79 ignored=0} inserts{seen=710 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:18.941704 23346 tablet_copy_service.cc:342] P ad9452840e4e4184a640fb63dab754fd: Request end of tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-b1a0cebcdad24e34abc11eef9d4d891d received from {username='slave'} at 127.20.210.7:35207
I20240501 19:56:18.941792 23346 tablet_copy_service.cc:433] P ad9452840e4e4184a640fb63dab754fd: ending tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-b1a0cebcdad24e34abc11eef9d4d891d on tablet b1a0cebcdad24e34abc11eef9d4d891d with peer 3b3f502cbeb1451087a06f95a3d93f2b
I20240501 19:56:18.943888 25113 tablet_bootstrap.cc:492] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap complete.
I20240501 19:56:18.944056 25113 ts_tablet_manager.cc:1367] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent bootstrapping tablet: real 0.473s user 0.067s sys 0.009s
I20240501 19:56:18.944470 25113 raft_consensus.cc:348] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:18.944582 25113 raft_consensus.cc:729] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3b3f502cbeb1451087a06f95a3d93f2b, State: Initialized, Role: LEARNER
I20240501 19:56:18.944679 25113 consensus_queue.cc:260] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 79, Last appended: 1.79, Last appended by leader: 79, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:18.947110 25113 ts_tablet_manager.cc:1398] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:18.949527 23477 tablet_copy_service.cc:342] P 0496e037179e4b3ca72aead781c25d5a: Request end of tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-a4ef2fdbb40e48b18025136adb39defc received from {username='slave'} at 127.20.210.7:36339
I20240501 19:56:18.949589 23477 tablet_copy_service.cc:433] P 0496e037179e4b3ca72aead781c25d5a: ending tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-a4ef2fdbb40e48b18025136adb39defc on tablet a4ef2fdbb40e48b18025136adb39defc with peer 3b3f502cbeb1451087a06f95a3d93f2b
I20240501 19:56:18.952106 25107 tablet_bootstrap.cc:492] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a: Bootstrap replayed 1/1 log segments. Stats: ops{read=69 overwritten=0 applied=69 ignored=0} inserts{seen=741 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:18.952565 25107 tablet_bootstrap.cc:492] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a: Bootstrap complete.
I20240501 19:56:18.952740 25107 ts_tablet_manager.cc:1367] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a: Time spent bootstrapping tablet: real 0.503s user 0.061s sys 0.011s
I20240501 19:56:18.953133 25107 raft_consensus.cc:348] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: true } }
I20240501 19:56:18.953245 25107 raft_consensus.cc:729] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0496e037179e4b3ca72aead781c25d5a, State: Initialized, Role: LEARNER
I20240501 19:56:18.953325 25107 consensus_queue.cc:260] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 69, Last appended: 1.69, Last appended by leader: 69, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: true } }
I20240501 19:56:18.953953 25107 ts_tablet_manager.cc:1398] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:18.961894 23608 tablet_copy_service.cc:342] P e8ddd74cfd954c7a8c2d284c335d7826: Request end of tablet copy session 0496e037179e4b3ca72aead781c25d5a-af8e2ce72fb24a18b54b0d2e49383b41 received from {username='slave'} at 127.20.210.2:60845
I20240501 19:56:18.961978 23608 tablet_copy_service.cc:433] P e8ddd74cfd954c7a8c2d284c335d7826: ending tablet copy session 0496e037179e4b3ca72aead781c25d5a-af8e2ce72fb24a18b54b0d2e49383b41 on tablet af8e2ce72fb24a18b54b0d2e49383b41 with peer 0496e037179e4b3ca72aead781c25d5a
I20240501 19:56:18.962334 25178 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 82, Last known committed idx: 80, Time since last communication: 0.000s
I20240501 19:56:18.972528 23849 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.82: config changed from index 80 to 82, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 82 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.032634 24756 raft_consensus.cc:1203] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.65->[1.66-1.73] Dedup: 1.66->[1.67-1.73]
I20240501 19:56:19.087790 25178 raft_consensus.cc:1050] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd: attempting to promote NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b to VOTER
I20240501 19:56:19.088171 25178 consensus_queue.cc:237] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 73, Committed index: 73, Last appended: 1.73, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:19.090322 24756 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 19:56:19.090656 24902 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 19:56:19.091105 23976 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 19:56:19.092218 24519 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 72, Time since last communication: 0.000s
I20240501 19:56:19.095628 25146 tablet_bootstrap.cc:492] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=66 overwritten=0 applied=65 ignored=0} inserts{seen=696 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240501 19:56:19.096043 25146 tablet_bootstrap.cc:492] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap complete.
I20240501 19:56:19.096745 25146 ts_tablet_manager.cc:1367] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent bootstrapping tablet: real 0.161s user 0.064s sys 0.000s
I20240501 19:56:19.097214 25146 raft_consensus.cc:348] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:19.097932 25146 raft_consensus.cc:729] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3b3f502cbeb1451087a06f95a3d93f2b, State: Initialized, Role: LEARNER
I20240501 19:56:19.098101 25146 consensus_queue.cc:260] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 65, Last appended: 1.66, Last appended by leader: 66, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 61 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:19.100155 25146 ts_tablet_manager.cc:1398] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:19.102775 23846 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 19:56:19.103878 24519 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 19:56:19.119253 24519 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.74: config changed from index 66 to 74, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.121665 24756 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 66 to 74, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.121846 23978 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 66 to 74, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.122728 23184 catalog_manager.cc:5484] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index 66 to 74, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:19.130462 23850 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 66 to 74, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
W20240501 19:56:19.144490 23391 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.6:37667, user_credentials={real_user=slave}} blocked reactor thread for 66713.6us
I20240501 19:56:19.167800 23455 raft_consensus.cc:1203] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.48->[1.49-1.83] Dedup: 1.69->[1.70-1.83]
I20240501 19:56:19.169368 23454 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 19:56:19.176149 24395 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 19:56:19.176908 23739 tablet_copy_service.cc:342] P 6e47806645ae47278e35600cc67d5700: Request end of tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-9f0625a730b0408b963d94f24e25fed7 received from {username='slave'} at 127.20.210.7:54081
I20240501 19:56:19.176999 23739 tablet_copy_service.cc:433] P 6e47806645ae47278e35600cc67d5700: ending tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-9f0625a730b0408b963d94f24e25fed7 on tablet 9f0625a730b0408b963d94f24e25fed7 with peer 3b3f502cbeb1451087a06f95a3d93f2b
I20240501 19:56:19.198628 23454 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 66 to 74, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.209918 23718 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 74. (index mismatch)
I20240501 19:56:19.211422 24902 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 74, Last known committed idx: 73, Time since last communication: 0.000s
I20240501 19:56:19.234593 23718 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.74: config changed from index 66 to 74, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 74 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.256891 23325 consensus_queue.cc:237] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 77, Committed index: 77, Last appended: 1.77, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:19.260389 24756 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 77. Preceding OpId from leader: term: 1 index: 78. (index mismatch)
I20240501 19:56:19.260748 24902 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 78, Last known committed idx: 77, Time since last communication: 0.000s
I20240501 19:56:19.264015 23978 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 77. Preceding OpId from leader: term: 1 index: 78. (index mismatch)
I20240501 19:56:19.264366 24902 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 78, Last known committed idx: 77, Time since last communication: 0.000s
I20240501 19:56:19.268741 24902 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.78: config changed from index 74 to 78, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.271029 23170 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b1a0cebcdad24e34abc11eef9d4d891d with cas_config_opid_index 74: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:19.273684 24756 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.78: config changed from index 74 to 78, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.274771 23184 catalog_manager.cc:5484] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index 74 to 78, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:19.279089 23830 tablet_service.cc:1514] Processing DeleteTablet for tablet b1a0cebcdad24e34abc11eef9d4d891d with delete_type TABLET_DATA_TOMBSTONED (TS ee9bc706484e48fb8ff739a61cabe858 not found in new config with opid_index 78) from {username='slave'} at 127.0.0.1:57012
I20240501 19:56:19.281396 23976 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.78: config changed from index 74 to 78, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.288219 25205 tablet_replica.cc:330] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858: stopping tablet replica
I20240501 19:56:19.288556 25205 raft_consensus.cc:2229] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:19.289069 25205 raft_consensus.cc:2258] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:19.292548 25205 ts_tablet_manager.cc:1875] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:19.294984 24969 raft_consensus.cc:1050] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826: attempting to promote NON_VOTER 0496e037179e4b3ca72aead781c25d5a to VOTER
I20240501 19:56:19.295481 24969 consensus_queue.cc:237] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 87, Committed index: 87, Last appended: 1.87, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 88 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } }
I20240501 19:56:19.296247 25205 ts_tablet_manager.cc:1888] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.77
I20240501 19:56:19.296382 25205 log.cc:1196] T b1a0cebcdad24e34abc11eef9d4d891d P ee9bc706484e48fb8ff739a61cabe858: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/wals/b1a0cebcdad24e34abc11eef9d4d891d
I20240501 19:56:19.297240 23716 raft_consensus.cc:1261] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer e8ddd74cfd954c7a8c2d284c335d7826: Log matching property violated. Preceding OpId in replica: term: 1 index: 87. Preceding OpId from leader: term: 1 index: 88. (index mismatch)
I20240501 19:56:19.297245 23168 catalog_manager.cc:4830] TS ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723): tablet b1a0cebcdad24e34abc11eef9d4d891d (table rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]) successfully deleted
I20240501 19:56:19.297688 25071 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 88, Last known committed idx: 87, Time since last communication: 0.000s
I20240501 19:56:19.297832 23455 raft_consensus.cc:1261] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEARNER]: Refusing update from remote peer e8ddd74cfd954c7a8c2d284c335d7826: Log matching property violated. Preceding OpId in replica: term: 1 index: 87. Preceding OpId from leader: term: 1 index: 88. (index mismatch)
I20240501 19:56:19.298269 25071 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 88, Last known committed idx: 87, Time since last communication: 0.000s
I20240501 19:56:19.299304 23976 raft_consensus.cc:1261] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer e8ddd74cfd954c7a8c2d284c335d7826: Log matching property violated. Preceding OpId in replica: term: 1 index: 87. Preceding OpId from leader: term: 1 index: 88. (index mismatch)
I20240501 19:56:19.301652 25071 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 88, Last known committed idx: 87, Time since last communication: 0.000s
I20240501 19:56:19.302438 23850 raft_consensus.cc:1261] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer e8ddd74cfd954c7a8c2d284c335d7826: Log matching property violated. Preceding OpId in replica: term: 1 index: 87. Preceding OpId from leader: term: 1 index: 88. (index mismatch)
I20240501 19:56:19.303097 25071 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 88, Last known committed idx: 87, Time since last communication: 0.000s
I20240501 19:56:19.310937 23325 raft_consensus.cc:1261] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer e8ddd74cfd954c7a8c2d284c335d7826: Log matching property violated. Preceding OpId in replica: term: 1 index: 87. Preceding OpId from leader: term: 1 index: 88. (index mismatch)
I20240501 19:56:19.311336 25071 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 88, Last known committed idx: 87, Time since last communication: 0.000s
I20240501 19:56:19.313791 24882 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 LEADER]: Committing config change with OpId 1.88: config changed from index 49 to 88, 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 88 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } } }
I20240501 19:56:19.315979 23849 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.88: config changed from index 49 to 88, 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 88 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } } }
I20240501 19:56:19.316220 23716 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.88: config changed from index 49 to 88, 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 88 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } } }
I20240501 19:56:19.317461 23323 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.88: config changed from index 49 to 88, 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 88 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } } }
I20240501 19:56:19.318111 23184 catalog_manager.cc:5484] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index 49 to 88, 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" committed_config { opid_index: 88 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } } }
I20240501 19:56:19.321707 23981 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.88: config changed from index 49 to 88, 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 88 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } } }
I20240501 19:56:19.328302 23452 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 76. Preceding OpId from leader: term: 1 index: 78. (index mismatch)
I20240501 19:56:19.328953 23717 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 77. Preceding OpId from leader: term: 1 index: 78. (index mismatch)
I20240501 19:56:19.333595 25023 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 78, Last known committed idx: 76, Time since last communication: 0.000s
I20240501 19:56:19.333930 25023 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 78, Last known committed idx: 76, Time since last communication: 0.000s
I20240501 19:56:19.336035 23456 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.78: config changed from index 74 to 78, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.348587 23718 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.78: config changed from index 74 to 78, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.354887 23586 consensus_queue.cc:237] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 89, Committed index: 89, Last appended: 1.89, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 90 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } }
I20240501 19:56:19.355096 23455 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.88: config changed from index 49 to 88, 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 88 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } } }
I20240501 19:56:19.356340 23452 raft_consensus.cc:1261] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer e8ddd74cfd954c7a8c2d284c335d7826: Log matching property violated. Preceding OpId in replica: term: 1 index: 88. Preceding OpId from leader: term: 1 index: 90. (index mismatch)
I20240501 19:56:19.356693 25071 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 90, Last known committed idx: 88, Time since last communication: 0.000s
I20240501 19:56:19.358757 23323 raft_consensus.cc:1261] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer e8ddd74cfd954c7a8c2d284c335d7826: Log matching property violated. Preceding OpId in replica: term: 1 index: 89. Preceding OpId from leader: term: 1 index: 90. (index mismatch)
I20240501 19:56:19.359678 25071 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 90, Last known committed idx: 89, Time since last communication: 0.000s
I20240501 19:56:19.362978 23849 raft_consensus.cc:1261] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer e8ddd74cfd954c7a8c2d284c335d7826: Log matching property violated. Preceding OpId in replica: term: 1 index: 89. Preceding OpId from leader: term: 1 index: 90. (index mismatch)
I20240501 19:56:19.363915 25071 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 90, Last known committed idx: 89, Time since last communication: 0.000s
I20240501 19:56:19.365595 24756 raft_consensus.cc:1203] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.73->[1.74-1.93] Dedup: 1.79->[1.80-1.93]
I20240501 19:56:19.366611 25071 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 LEADER]: Committing config change with OpId 1.90: config changed from index 88 to 90, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 90 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } } }
I20240501 19:56:19.368353 23171 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet af8e2ce72fb24a18b54b0d2e49383b41 with cas_config_opid_index 88: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:19.370069 23850 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.90: config changed from index 88 to 90, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 90 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } } }
I20240501 19:56:19.373850 23323 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.90: config changed from index 88 to 90, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 90 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } } }
I20240501 19:56:19.376962 23182 catalog_manager.cc:5484] T af8e2ce72fb24a18b54b0d2e49383b41 P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index 88 to 90, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" committed_config { opid_index: 90 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } } }
I20240501 19:56:19.379376 23698 tablet_service.cc:1514] Processing DeleteTablet for tablet af8e2ce72fb24a18b54b0d2e49383b41 with delete_type TABLET_DATA_TOMBSTONED (TS 6e47806645ae47278e35600cc67d5700 not found in new config with opid_index 90) from {username='slave'} at 127.0.0.1:54842
I20240501 19:56:19.379787 25211 tablet_replica.cc:330] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700: stopping tablet replica
I20240501 19:56:19.380299 25211 raft_consensus.cc:2229] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:19.380481 25211 raft_consensus.cc:2258] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:19.382231 25211 ts_tablet_manager.cc:1875] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:19.387516 25211 ts_tablet_manager.cc:1888] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.89
I20240501 19:56:19.387641 25211 log.cc:1196] T af8e2ce72fb24a18b54b0d2e49383b41 P 6e47806645ae47278e35600cc67d5700: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/af8e2ce72fb24a18b54b0d2e49383b41
I20240501 19:56:19.388394 23171 catalog_manager.cc:4830] TS 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657): tablet af8e2ce72fb24a18b54b0d2e49383b41 (table rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]) successfully deleted
I20240501 19:56:19.389640 23456 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.90: config changed from index 88 to 90, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 90 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } } }
I20240501 19:56:19.397485 23582 raft_consensus.cc:1203] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.60->[1.61-1.77] Dedup: 1.63->[1.64-1.77]
I20240501 19:56:19.411058 23978 raft_consensus.cc:1261] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer e8ddd74cfd954c7a8c2d284c335d7826: Log matching property violated. Preceding OpId in replica: term: 1 index: 89. Preceding OpId from leader: term: 1 index: 90. (index mismatch)
I20240501 19:56:19.423106 25207 consensus_queue.cc:1035] T af8e2ce72fb24a18b54b0d2e49383b41 P e8ddd74cfd954c7a8c2d284c335d7826 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 90, Last known committed idx: 89, Time since last communication: 0.000s
I20240501 19:56:19.423998 24756 raft_consensus.cc:1203] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.60->[1.61-1.69] Dedup: 1.66->[1.67-1.69]
I20240501 19:56:19.430928 23978 raft_consensus.cc:2932] T af8e2ce72fb24a18b54b0d2e49383b41 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.90: config changed from index 88 to 90, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 90 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { promote: false } } }
I20240501 19:56:19.442776 24849 raft_consensus.cc:1050] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700: attempting to promote NON_VOTER e8ddd74cfd954c7a8c2d284c335d7826 to VOTER
I20240501 19:56:19.446199 24849 consensus_queue.cc:237] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 77, Committed index: 77, Last appended: 1.77, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } }
I20240501 19:56:19.448297 23582 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 LEARNER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 77. Preceding OpId from leader: term: 1 index: 78. (index mismatch)
I20240501 19:56:19.453945 24971 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 78, Last known committed idx: 77, Time since last communication: 0.000s
I20240501 19:56:19.457995 23325 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 77. Preceding OpId from leader: term: 1 index: 78. (index mismatch)
I20240501 19:56:19.458231 23849 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 77. Preceding OpId from leader: term: 1 index: 78. (index mismatch)
I20240501 19:56:19.458685 23455 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 77. Preceding OpId from leader: term: 1 index: 78. (index mismatch)
I20240501 19:56:19.460881 25155 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 78, Last known committed idx: 76, Time since last communication: 0.000s
I20240501 19:56:19.461867 24540 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 78, Last known committed idx: 76, Time since last communication: 0.000s
I20240501 19:56:19.470918 24540 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 78, Last known committed idx: 77, Time since last communication: 0.000s
I20240501 19:56:19.485599 25073 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.78: config changed from index 61 to 78, e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } }
I20240501 19:56:19.488128 23585 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.78: config changed from index 61 to 78, e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } }
I20240501 19:56:19.493173 23182 catalog_manager.cc:5484] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 reported cstate change: config changed from index 61 to 78, e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:19.496369 25218 raft_consensus.cc:1050] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700: attempting to promote NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b to VOTER
I20240501 19:56:19.496709 25218 consensus_queue.cc:237] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 69, Committed index: 69, Last appended: 1.70, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 71 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:19.497740 23321 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.78: config changed from index 61 to 78, e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } }
I20240501 19:56:19.502490 23586 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 70. Preceding OpId from leader: term: 1 index: 71. (index mismatch)
I20240501 19:56:19.502655 23978 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 77. Preceding OpId from leader: term: 1 index: 78. (index mismatch)
I20240501 19:56:19.503072 25155 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 71, Last known committed idx: 69, Time since last communication: 0.000s
I20240501 19:56:19.505074 24756 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 70. Preceding OpId from leader: term: 1 index: 71. (index mismatch)
I20240501 19:56:19.505964 24540 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 71, Last known committed idx: 69, Time since last communication: 0.000s
I20240501 19:56:19.505916 25218 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 78, Last known committed idx: 76, Time since last communication: 0.000s
I20240501 19:56:19.508860 23321 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 70. Preceding OpId from leader: term: 1 index: 71. (index mismatch)
I20240501 19:56:19.511729 23976 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 70. Preceding OpId from leader: term: 1 index: 71. (index mismatch)
I20240501 19:56:19.513995 23455 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.78: config changed from index 61 to 78, e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } }
I20240501 19:56:19.519129 23455 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 70. Preceding OpId from leader: term: 1 index: 71. (index mismatch)
I20240501 19:56:19.525498 25218 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 71, Last known committed idx: 69, Time since last communication: 0.000s
I20240501 19:56:19.527469 25218 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 71, Last known committed idx: 69, Time since last communication: 0.000s
I20240501 19:56:19.532815 25218 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.71: config changed from index 61 to 71, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 71 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.534813 23977 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.71: config changed from index 61 to 71, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 71 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.534883 23588 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.71: config changed from index 61 to 71, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 71 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.536571 24756 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.71: config changed from index 61 to 71, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 71 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.537627 23182 catalog_manager.cc:5484] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 reported cstate change: config changed from index 61 to 71, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 71 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.539335 23322 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.71: config changed from index 61 to 71, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 71 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.531802 23718 consensus_queue.cc:237] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 78, Committed index: 78, Last appended: 1.79, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } }
I20240501 19:56:19.551358 23586 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 79. Preceding OpId from leader: term: 1 index: 80. (index mismatch)
I20240501 19:56:19.551640 25073 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 80, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 19:56:19.554569 23325 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 79. Preceding OpId from leader: term: 1 index: 80. (index mismatch)
I20240501 19:56:19.558522 23976 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.78: config changed from index 61 to 78, e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } }
I20240501 19:56:19.562176 24849 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 80, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 19:56:19.563992 23849 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.78: config changed from index 61 to 78, e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 78 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } }
I20240501 19:56:19.571595 24849 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.80: config changed from index 78 to 80, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } }
I20240501 19:56:19.573793 23171 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6f549fed054d433381ac6d38d64c74bb with cas_config_opid_index 78: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:19.573827 23325 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.80: config changed from index 78 to 80, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } }
I20240501 19:56:19.573819 23588 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.80: config changed from index 78 to 80, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } }
I20240501 19:56:19.576346 23849 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 79. Preceding OpId from leader: term: 1 index: 80. (index mismatch)
I20240501 19:56:19.576565 23182 catalog_manager.cc:5484] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index 78 to 80, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } }
I20240501 19:56:19.577872 23455 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 78. Preceding OpId from leader: term: 1 index: 80. (index mismatch)
I20240501 19:56:19.578871 23961 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f549fed054d433381ac6d38d64c74bb with delete_type TABLET_DATA_TOMBSTONED (TS 4f51f982ad284c22aae01c4541a621f1 not found in new config with opid_index 80) from {username='slave'} at 127.0.0.1:45934
I20240501 19:56:19.596724 25225 tablet_replica.cc:330] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1: stopping tablet replica
I20240501 19:56:19.597162 25225 raft_consensus.cc:2229] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:19.599066 25225 pending_rounds.cc:62] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1: Trying to abort 1 pending ops.
I20240501 19:56:19.599184 25225 pending_rounds.cc:68] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1: Aborting op as it isn't in flight: id { term: 1 index: 79 } timestamp: 7022974482431930368 op_type: WRITE_OP write_request { tablet_id: "6f549fed054d433381ac6d38d64c74bb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000?U\253 t\236\221I\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""3f50e42baf6808b4" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7022974482329681920 authz_token { token_data: "\010\212\277\312\261\006\"3\n\005slave\022*\n 40a9a76ff0f04080a591a2a11c721af1\020\001\030\001 \001(\001" signature: "<redacted>""\210BD\211\237vN\315@<\214S\032\241\016\326oL\233\322\260E\346\241\365`\217\222\241\317\310Xm\375]\335\2318\362\237)]\n+\310\347\363\340\033\363Br@\016\323\004(\032U\3507\211f\010" signing_key_seq_num: 0 } } request_id { client_id: "2225eb81fa694670b52e08f1917afa88" seq_no: 238 first_incomplete_seq_no: 238 attempt_no: 0 }
I20240501 19:56:19.604926 25073 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 71, Last known committed idx: 69, Time since last communication: 0.000s
I20240501 19:56:19.611729 25073 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 80, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 19:56:19.615204 25225 raft_consensus.cc:2258] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:19.621199 25225 ts_tablet_manager.cc:1875] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:19.624496 23454 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.80: config changed from index 78 to 80, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } }
I20240501 19:56:19.624855 23456 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.71: config changed from index 61 to 71, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 71 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.627291 25225 ts_tablet_manager.cc:1888] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.79
I20240501 19:56:19.627427 25225 log.cc:1196] T 6f549fed054d433381ac6d38d64c74bb P 4f51f982ad284c22aae01c4541a621f1: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/wals/6f549fed054d433381ac6d38d64c74bb
I20240501 19:56:19.657790 23169 catalog_manager.cc:4830] TS 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667): tablet 6f549fed054d433381ac6d38d64c74bb (table rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]) successfully deleted
I20240501 19:56:19.667882 25073 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 80, Last known committed idx: 78, Time since last communication: 0.000s
I20240501 19:56:19.668743 24988 raft_consensus.cc:1050] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a: attempting to promote NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b to VOTER
I20240501 19:56:19.669054 24988 consensus_queue.cc:237] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 94, Committed index: 94, Last appended: 1.94, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 95 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:19.671259 23716 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 93. Preceding OpId from leader: term: 1 index: 95. (index mismatch)
I20240501 19:56:19.672374 23849 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 94. Preceding OpId from leader: term: 1 index: 95. (index mismatch)
I20240501 19:56:19.672799 24926 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 95, Last known committed idx: 93, Time since last communication: 0.000s
I20240501 19:56:19.673146 24926 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 95, Last known committed idx: 93, Time since last communication: 0.000s
I20240501 19:56:19.677803 24756 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 94. Preceding OpId from leader: term: 1 index: 95. (index mismatch)
I20240501 19:56:19.678694 24927 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 95, Last known committed idx: 93, Time since last communication: 0.000s
I20240501 19:56:19.679828 23979 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 94. Preceding OpId from leader: term: 1 index: 95. (index mismatch)
I20240501 19:56:19.683801 24927 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 95, Last known committed idx: 94, Time since last communication: 0.000s
I20240501 19:56:19.687104 23584 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 94. Preceding OpId from leader: term: 1 index: 95. (index mismatch)
I20240501 19:56:19.688392 24926 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 95, Last known committed idx: 94, Time since last communication: 0.000s
I20240501 19:56:19.688910 24927 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Committing config change with OpId 1.95: config changed from index 74 to 95, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 95 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.689613 24757 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.95: config changed from index 74 to 95, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 95 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.690883 23716 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.95: config changed from index 74 to 95, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 95 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.691319 23184 catalog_manager.cc:5484] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a reported cstate change: config changed from index 74 to 95, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: 95 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:19.692020 23850 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.95: config changed from index 74 to 95, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 95 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.693396 23587 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.95: config changed from index 74 to 95, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 95 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.695381 23846 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.80: config changed from index 78 to 80, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 80 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } }
I20240501 19:56:19.706358 23978 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.95: config changed from index 74 to 95, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 95 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.709873 23454 consensus_queue.cc:237] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 95, Committed index: 95, Last appended: 1.95, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:19.712034 23849 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 95. Preceding OpId from leader: term: 1 index: 96. (index mismatch)
I20240501 19:56:19.712335 25042 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 96, Last known committed idx: 95, Time since last communication: 0.000s
I20240501 19:56:19.714694 23586 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 95. Preceding OpId from leader: term: 1 index: 96. (index mismatch)
I20240501 19:56:19.715066 24926 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 96, Last known committed idx: 95, Time since last communication: 0.000s
I20240501 19:56:19.717535 24926 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Committing config change with OpId 1.96: config changed from index 95 to 96, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.719359 23588 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.96: config changed from index 95 to 96, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.720181 24757 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 95. Preceding OpId from leader: term: 1 index: 96. (index mismatch)
I20240501 19:56:19.720405 23978 raft_consensus.cc:1261] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 95. Preceding OpId from leader: term: 1 index: 96. (index mismatch)
I20240501 19:56:19.720780 23168 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a4ef2fdbb40e48b18025136adb39defc with cas_config_opid_index 95: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:19.724170 23849 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.96: config changed from index 95 to 96, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.726018 23184 catalog_manager.cc:5484] T a4ef2fdbb40e48b18025136adb39defc P e8ddd74cfd954c7a8c2d284c335d7826 reported cstate change: config changed from index 95 to 96, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.728392 23698 tablet_service.cc:1514] Processing DeleteTablet for tablet a4ef2fdbb40e48b18025136adb39defc with delete_type TABLET_DATA_TOMBSTONED (TS 6e47806645ae47278e35600cc67d5700 not found in new config with opid_index 96) from {username='slave'} at 127.0.0.1:54842
I20240501 19:56:19.731508 24988 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 96, Last known committed idx: 95, Time since last communication: 0.000s
I20240501 19:56:19.731742 24988 consensus_queue.cc:1035] T a4ef2fdbb40e48b18025136adb39defc P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 96, Last known committed idx: 95, Time since last communication: 0.000s
I20240501 19:56:19.734427 24756 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.96: config changed from index 95 to 96, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.735352 23978 raft_consensus.cc:2932] T a4ef2fdbb40e48b18025136adb39defc P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.96: config changed from index 95 to 96, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 96 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.738018 25211 tablet_replica.cc:330] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700: stopping tablet replica
I20240501 19:56:19.738142 25211 raft_consensus.cc:2229] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:19.738421 25211 raft_consensus.cc:2258] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:19.739351 25211 ts_tablet_manager.cc:1875] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:19.741269 25211 ts_tablet_manager.cc:1888] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.95
I20240501 19:56:19.741349 25211 log.cc:1196] T a4ef2fdbb40e48b18025136adb39defc P 6e47806645ae47278e35600cc67d5700: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/a4ef2fdbb40e48b18025136adb39defc
I20240501 19:56:19.741930 23171 catalog_manager.cc:4830] TS 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657): tablet a4ef2fdbb40e48b18025136adb39defc (table rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]) successfully deleted
I20240501 19:56:19.748812 23698 tablet_service.cc:1514] Processing DeleteTablet for tablet a4ef2fdbb40e48b18025136adb39defc with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 95 (current committed config index is 96)) from {username='slave'} at 127.0.0.1:54842
I20240501 19:56:19.750098 23171 catalog_manager.cc:4830] TS 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657): tablet a4ef2fdbb40e48b18025136adb39defc (table rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]) successfully deleted
I20240501 19:56:19.754516 23717 consensus_queue.cc:237] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 74, Committed index: 74, Last appended: 1.74, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:19.755764 24756 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 74. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240501 19:56:19.756042 23588 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 74. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240501 19:56:19.756340 24540 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 75, Last known committed idx: 74, Time since last communication: 0.000s
I20240501 19:56:19.756409 23977 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 74. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240501 19:56:19.756594 24540 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 75, Last known committed idx: 74, Time since last communication: 0.000s
I20240501 19:56:19.759768 24540 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.75: config changed from index 71 to 75, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.759927 25155 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 75, Last known committed idx: 74, Time since last communication: 0.000s
I20240501 19:56:19.761346 23171 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9f0625a730b0408b963d94f24e25fed7 with cas_config_opid_index 71: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:19.763130 23584 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.75: config changed from index 71 to 75, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.764165 23978 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.75: config changed from index 71 to 75, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.762645 23184 catalog_manager.cc:5484] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 reported cstate change: config changed from index 71 to 75, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:19.767971 24756 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.75: config changed from index 71 to 75, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.776567 23305 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f0625a730b0408b963d94f24e25fed7 with delete_type TABLET_DATA_TOMBSTONED (TS ad9452840e4e4184a640fb63dab754fd not found in new config with opid_index 75) from {username='slave'} at 127.0.0.1:55042
W20240501 19:56:19.827291 23652 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.2:38367, user_credentials={real_user=slave}} blocked reactor thread for 49318us
I20240501 19:56:19.827807 23455 raft_consensus.cc:1261] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 74. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240501 19:56:19.861550 24540 consensus_queue.cc:1035] T 9f0625a730b0408b963d94f24e25fed7 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 75, Last known committed idx: 74, Time since last communication: 0.000s
I20240501 19:56:19.865283 23455 raft_consensus.cc:2932] T 9f0625a730b0408b963d94f24e25fed7 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.75: config changed from index 71 to 75, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:19.874925 25228 tablet_replica.cc:330] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd: stopping tablet replica
I20240501 19:56:19.878105 25228 raft_consensus.cc:2229] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:19.893972 25228 raft_consensus.cc:2258] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:19.897591 25228 ts_tablet_manager.cc:1875] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:19.972795 25228 ts_tablet_manager.cc:1888] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.74
I20240501 19:56:19.984546 25228 log.cc:1196] T 9f0625a730b0408b963d94f24e25fed7 P ad9452840e4e4184a640fb63dab754fd: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/9f0625a730b0408b963d94f24e25fed7
I20240501 19:56:19.985153 23170 catalog_manager.cc:4830] TS ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481): tablet 9f0625a730b0408b963d94f24e25fed7 (table rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]) successfully deleted
I20240501 19:56:20.045092 23718 consensus_queue.cc:237] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 109, Committed index: 109, Last appended: 1.109, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
W20240501 19:56:20.046844 23653 consensus_peers.cc:485] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 -> Peer 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019): Couldn't send request to peer 3b3f502cbeb1451087a06f95a3d93f2b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 292aa245734c41d9814b8347d543b8fb. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:20.047554 23587 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 109. Preceding OpId from leader: term: 1 index: 110. (index mismatch)
I20240501 19:56:20.047878 24848 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 110, Last known committed idx: 109, Time since last communication: 0.000s
I20240501 19:56:20.050621 23981 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 109. Preceding OpId from leader: term: 1 index: 110. (index mismatch)
I20240501 19:56:20.052031 23850 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 109. Preceding OpId from leader: term: 1 index: 110. (index mismatch)
I20240501 19:56:20.052212 25218 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 110, Last known committed idx: 109, Time since last communication: 0.000s
I20240501 19:56:20.055773 23455 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 109. Preceding OpId from leader: term: 1 index: 110. (index mismatch)
I20240501 19:56:20.066653 25218 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 110, Last known committed idx: 109, Time since last communication: 0.000s
I20240501 19:56:20.074286 25218 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 110, Last known committed idx: 109, Time since last communication: 0.000s
I20240501 19:56:20.081889 24971 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.110: config changed from index -1 to 110, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:20.084625 23184 catalog_manager.cc:5484] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 reported cstate change: config changed from index -1 to 110, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:20.087431 23977 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.110: config changed from index -1 to 110, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:20.088297 23455 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.110: config changed from index -1 to 110, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:20.090001 23583 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.110: config changed from index -1 to 110, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:20.106427 23850 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.110: config changed from index -1 to 110, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:20.307296 23849 consensus_queue.cc:237] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 130, Committed index: 130, Last appended: 1.130, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 131 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:20.311057 23716 raft_consensus.cc:1261] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 130. Preceding OpId from leader: term: 1 index: 131. (index mismatch)
I20240501 19:56:20.313949 23583 raft_consensus.cc:1261] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 130. Preceding OpId from leader: term: 1 index: 131. (index mismatch)
I20240501 19:56:20.316962 23974 raft_consensus.cc:1261] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 130. Preceding OpId from leader: term: 1 index: 131. (index mismatch)
I20240501 19:56:20.320343 25156 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 131, Last known committed idx: 130, Time since last communication: 0.000s
I20240501 19:56:20.323508 24392 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 131, Last known committed idx: 130, Time since last communication: 0.000s
I20240501 19:56:20.323851 24392 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 131, Last known committed idx: 129, Time since last communication: 0.000s
I20240501 19:56:20.328003 25156 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Committing config change with OpId 1.131: config changed from index -1 to 131, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 131 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:20.329262 23716 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.131: config changed from index -1 to 131, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 131 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:20.332434 23184 catalog_manager.cc:5484] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index -1 to 131, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: 131 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:20.334103 23979 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.131: config changed from index -1 to 131, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 131 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:20.349517 23587 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.131: config changed from index -1 to 131, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 131 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
W20240501 19:56:20.368508 23784 consensus_peers.cc:485] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 -> Peer 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019): Couldn't send request to peer 3b3f502cbeb1451087a06f95a3d93f2b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2eea4673ba94b0fa5a2e0f099532777. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:20.416195 23321 raft_consensus.cc:1261] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 130. Preceding OpId from leader: term: 1 index: 131. (index mismatch)
I20240501 19:56:20.450817 24837 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 131, Last known committed idx: 130, Time since last communication: 0.000s
I20240501 19:56:20.487960 23321 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.131: config changed from index -1 to 131, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 131 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:20.512053 25252 ts_tablet_manager.cc:902] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b: Initiating tablet copy from peer 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657)
I20240501 19:56:20.514633 25252 tablet_copy_client.cc:308] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:44657
I20240501 19:56:20.515209 23739 tablet_copy_service.cc:140] P 6e47806645ae47278e35600cc67d5700: Received BeginTabletCopySession request for tablet 292aa245734c41d9814b8347d543b8fb from peer 3b3f502cbeb1451087a06f95a3d93f2b ({username='slave'} at 127.20.210.7:54081)
I20240501 19:56:20.515305 23739 tablet_copy_service.cc:161] P 6e47806645ae47278e35600cc67d5700: Beginning new tablet copy session on tablet 292aa245734c41d9814b8347d543b8fb from peer 3b3f502cbeb1451087a06f95a3d93f2b at {username='slave'} at 127.20.210.7:54081: session id = 3b3f502cbeb1451087a06f95a3d93f2b-292aa245734c41d9814b8347d543b8fb
I20240501 19:56:20.516301 23739 tablet_copy_source_session.cc:205] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:20.520449 25252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292aa245734c41d9814b8347d543b8fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:20.535455 25252 tablet_copy_client.cc:783] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:20.535742 25252 tablet_copy_client.cc:647] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:20.539479 25252 tablet_copy_client.cc:523] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:20.547629 25252 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap starting.
I20240501 19:56:20.800256 25266 ts_tablet_manager.cc:902] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b: Initiating tablet copy from peer ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:20.800916 25266 tablet_copy_client.cc:308] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.5:44723
I20240501 19:56:20.903002 25252 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=116 overwritten=0 applied=116 ignored=0} inserts{seen=747 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:20.903353 25252 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap complete.
I20240501 19:56:20.903563 25252 ts_tablet_manager.cc:1367] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent bootstrapping tablet: real 0.356s user 0.095s sys 0.012s
I20240501 19:56:20.905463 25252 raft_consensus.cc:348] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:20.905607 25252 raft_consensus.cc:729] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3b3f502cbeb1451087a06f95a3d93f2b, State: Initialized, Role: LEARNER
I20240501 19:56:20.905735 25252 consensus_queue.cc:260] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 116, Last appended: 1.116, Last appended by leader: 116, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 110 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:20.906116 25252 ts_tablet_manager.cc:1398] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:20.907344 23739 tablet_copy_service.cc:342] P 6e47806645ae47278e35600cc67d5700: Request end of tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-292aa245734c41d9814b8347d543b8fb received from {username='slave'} at 127.20.210.7:54081
I20240501 19:56:20.907420 23739 tablet_copy_service.cc:433] P 6e47806645ae47278e35600cc67d5700: ending tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-292aa245734c41d9814b8347d543b8fb on tablet 292aa245734c41d9814b8347d543b8fb with peer 3b3f502cbeb1451087a06f95a3d93f2b
I20240501 19:56:20.920341 23870 tablet_copy_service.cc:140] P ee9bc706484e48fb8ff739a61cabe858: Received BeginTabletCopySession request for tablet b2eea4673ba94b0fa5a2e0f099532777 from peer 3b3f502cbeb1451087a06f95a3d93f2b ({username='slave'} at 127.20.210.7:46877)
I20240501 19:56:20.920495 23870 tablet_copy_service.cc:161] P ee9bc706484e48fb8ff739a61cabe858: Beginning new tablet copy session on tablet b2eea4673ba94b0fa5a2e0f099532777 from peer 3b3f502cbeb1451087a06f95a3d93f2b at {username='slave'} at 127.20.210.7:46877: session id = 3b3f502cbeb1451087a06f95a3d93f2b-b2eea4673ba94b0fa5a2e0f099532777
I20240501 19:56:20.921970 23870 tablet_copy_source_session.cc:205] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:20.924120 25266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2eea4673ba94b0fa5a2e0f099532777. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:20.926074 25266 tablet_copy_client.cc:783] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:20.926209 25266 tablet_copy_client.cc:647] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:20.930202 25266 tablet_copy_client.cc:523] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:20.932775 25266 tablet_bootstrap.cc:492] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap starting.
I20240501 19:56:21.126500 25266 tablet_bootstrap.cc:492] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=147 overwritten=0 applied=147 ignored=0} inserts{seen=832 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:21.126849 25266 tablet_bootstrap.cc:492] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap complete.
I20240501 19:56:21.127571 25266 ts_tablet_manager.cc:1367] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent bootstrapping tablet: real 0.195s user 0.126s sys 0.000s
I20240501 19:56:21.128070 25266 raft_consensus.cc:348] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 131 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:21.129164 25266 raft_consensus.cc:729] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3b3f502cbeb1451087a06f95a3d93f2b, State: Initialized, Role: LEARNER
I20240501 19:56:21.129294 25266 consensus_queue.cc:260] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 147, Last appended: 1.147, Last appended by leader: 147, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 131 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:21.129604 25266 ts_tablet_manager.cc:1398] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:21.134137 23870 tablet_copy_service.cc:342] P ee9bc706484e48fb8ff739a61cabe858: Request end of tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-b2eea4673ba94b0fa5a2e0f099532777 received from {username='slave'} at 127.20.210.7:46877
I20240501 19:56:21.134230 23870 tablet_copy_service.cc:433] P ee9bc706484e48fb8ff739a61cabe858: ending tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-b2eea4673ba94b0fa5a2e0f099532777 on tablet b2eea4673ba94b0fa5a2e0f099532777 with peer 3b3f502cbeb1451087a06f95a3d93f2b
I20240501 19:56:21.212211 23846 consensus_queue.cc:237] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:21.214541 23980 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:21.214674 23454 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240501 19:56:21.225219 23784 consensus_peers.cc:485] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 -> Peer 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019): Couldn't send request to peer 3b3f502cbeb1451087a06f95a3d93f2b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 44f96d2d427d43008630eb7bcc39caf9. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:21.225332 24837 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 19:56:21.225592 24837 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 19:56:21.237093 24837 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.238629 23719 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:21.239374 23977 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.240221 23324 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:21.240113 23184 catalog_manager.cc:5484] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index -1 to 2, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:21.242616 23457 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.244486 24934 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 19:56:21.258651 24934 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240501 19:56:21.260809 23325 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.264106 23717 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.278168 23975 consensus_queue.cc:237] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 157, Committed index: 157, Last appended: 1.157, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 158 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:21.281414 24757 raft_consensus.cc:1203] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.130->[1.131-1.150] Dedup: 1.147->[1.148-1.150]
I20240501 19:56:21.283068 23455 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 157. Preceding OpId from leader: term: 1 index: 158. (index mismatch)
I20240501 19:56:21.283818 24821 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 158, Last known committed idx: 157, Time since last communication: 0.000s
I20240501 19:56:21.284511 23846 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 157. Preceding OpId from leader: term: 1 index: 158. (index mismatch)
I20240501 19:56:21.285143 23587 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 157. Preceding OpId from leader: term: 1 index: 158. (index mismatch)
I20240501 19:56:21.285562 24827 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 158, Last known committed idx: 157, Time since last communication: 0.000s
I20240501 19:56:21.285751 24484 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 158, Last known committed idx: 157, Time since last communication: 0.000s
I20240501 19:56:21.289338 24827 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 1 LEADER]: Committing config change with OpId 1.158: config changed from index -1 to 158, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 158 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.292660 23324 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 157. Preceding OpId from leader: term: 1 index: 158. (index mismatch)
I20240501 19:56:21.293052 24484 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 158, Last known committed idx: 157, Time since last communication: 0.000s
I20240501 19:56:21.294767 23455 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.158: config changed from index -1 to 158, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 158 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.295500 23324 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.158: config changed from index -1 to 158, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 158 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.303606 23582 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.158: config changed from index -1 to 158, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 158 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.304224 23850 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.158: config changed from index -1 to 158, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 158 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.306447 23179 catalog_manager.cc:5484] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 reported cstate change: config changed from index -1 to 158, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "4f51f982ad284c22aae01c4541a621f1" committed_config { opid_index: 158 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.316825 24392 raft_consensus.cc:1050] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858: attempting to promote NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b to VOTER
I20240501 19:56:21.317333 24392 consensus_queue.cc:237] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 150, Committed index: 150, Last appended: 1.151, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:21.319864 24755 raft_consensus.cc:1261] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 19:56:21.320324 24934 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 150, Time since last communication: 0.000s
I20240501 19:56:21.323107 23975 raft_consensus.cc:1261] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 19:56:21.323743 24934 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 150, Time since last communication: 0.000s
I20240501 19:56:21.344928 23719 raft_consensus.cc:1261] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 19:56:21.348416 23582 raft_consensus.cc:1261] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 19:56:21.348425 23325 raft_consensus.cc:1261] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 19:56:21.352663 24934 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 150, Time since last communication: 0.000s
I20240501 19:56:21.364964 24392 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 150, Time since last communication: 0.000s
I20240501 19:56:21.368660 24392 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 150, Time since last communication: 0.000s
I20240501 19:56:21.370352 23849 consensus_queue.cc:237] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 150, Committed index: 150, Last appended: 1.151, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:21.370676 25288 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Committing config change with OpId 1.152: config changed from index 131 to 152, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
W20240501 19:56:21.372246 23784 consensus_peers.cc:485] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 -> Peer 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019): Couldn't send request to peer 3b3f502cbeb1451087a06f95a3d93f2b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 107c33af3fbf4edbb3d3b4ba880e4661. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:21.372864 24757 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 131 to 152, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.373647 23452 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 19:56:21.374094 25000 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 150, Time since last communication: 0.000s
I20240501 19:56:21.376070 23178 catalog_manager.cc:5484] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index 131 to 152, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:21.379243 23980 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 131 to 152, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.382411 23587 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 19:56:21.382460 23325 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 150. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 19:56:21.398869 23717 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240501 19:56:21.399763 23719 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 131 to 152, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.407213 23587 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 131 to 152, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.408452 23849 consensus_queue.cc:237] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 152, Committed index: 152, Last appended: 1.152, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:21.410306 23979 raft_consensus.cc:1261] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240501 19:56:21.410820 24755 raft_consensus.cc:1261] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240501 19:56:21.410799 24934 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 152, Time since last communication: 0.000s
I20240501 19:56:21.411057 24934 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 150, Time since last communication: 0.000s
I20240501 19:56:21.411440 25000 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 152, Time since last communication: 0.000s
I20240501 19:56:21.412853 23719 raft_consensus.cc:1261] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240501 19:56:21.414144 24392 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 150, Time since last communication: 0.000s
I20240501 19:56:21.417282 24392 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 150, Time since last communication: 0.000s
I20240501 19:56:21.419801 24392 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 152, Time since last communication: 0.000s
I20240501 19:56:21.420708 24934 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Committing config change with OpId 1.153: config changed from index 152 to 153, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.422533 23168 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b2eea4673ba94b0fa5a2e0f099532777 with cas_config_opid_index 152: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:21.423224 23180 catalog_manager.cc:5484] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index 152 to 153, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:21.423573 23975 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 152 to 153, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.425290 25156 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Committing config change with OpId 1.152: config changed from index -1 to 152, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.425803 23568 tablet_service.cc:1514] Processing DeleteTablet for tablet b2eea4673ba94b0fa5a2e0f099532777 with delete_type TABLET_DATA_TOMBSTONED (TS e8ddd74cfd954c7a8c2d284c335d7826 not found in new config with opid_index 153) from {username='slave'} at 127.0.0.1:46492
I20240501 19:56:21.424906 24755 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 152 to 153, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.430684 23452 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index -1 to 152, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.431144 23180 catalog_manager.cc:5484] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index -1 to 152, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:21.435158 23582 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index -1 to 152, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.443555 23325 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 131 to 152, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.447903 25339 tablet_replica.cc:330] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826: stopping tablet replica
I20240501 19:56:21.448581 25339 raft_consensus.cc:2229] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:21.451241 25339 raft_consensus.cc:2258] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:21.452353 25339 ts_tablet_manager.cc:1875] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:21.454479 23325 raft_consensus.cc:1261] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240501 19:56:21.454943 25339 ts_tablet_manager.cc:1888] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.152
I20240501 19:56:21.455070 25339 log.cc:1196] T b2eea4673ba94b0fa5a2e0f099532777 P e8ddd74cfd954c7a8c2d284c335d7826: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/b2eea4673ba94b0fa5a2e0f099532777
I20240501 19:56:21.455901 23717 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 152 to 153, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.456180 23719 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index -1 to 152, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.461161 24934 consensus_queue.cc:1035] T b2eea4673ba94b0fa5a2e0f099532777 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 152, Time since last communication: 0.000s
I20240501 19:56:21.467638 23324 raft_consensus.cc:2932] T b2eea4673ba94b0fa5a2e0f099532777 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 152 to 153, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.472712 23322 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index -1 to 152, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:21.473047 23171 catalog_manager.cc:4830] TS e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279): tablet b2eea4673ba94b0fa5a2e0f099532777 (table rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]) successfully deleted
I20240501 19:56:21.475334 24755 raft_consensus.cc:1203] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.109->[1.110-1.140] Dedup: 1.116->[1.117-1.140]
W20240501 19:56:21.547585 23915 consensus_peers.cc:485] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 -> Peer 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019): Couldn't send request to peer 3b3f502cbeb1451087a06f95a3d93f2b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 78b919c4db7745fd87151d8834866e26. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:21.590438 24849 raft_consensus.cc:1050] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700: attempting to promote NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b to VOTER
I20240501 19:56:21.592522 24849 consensus_queue.cc:237] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 144, Committed index: 144, Last appended: 1.144, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 145 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:21.596518 23979 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 144. Preceding OpId from leader: term: 1 index: 146. (index mismatch)
I20240501 19:56:21.597024 25155 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 145, Last known committed idx: 144, Time since last communication: 0.000s
I20240501 19:56:21.597762 23587 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 144. Preceding OpId from leader: term: 1 index: 146. (index mismatch)
I20240501 19:56:21.604346 23455 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 144. Preceding OpId from leader: term: 1 index: 146. (index mismatch)
I20240501 19:56:21.608542 25155 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 145, Last known committed idx: 144, Time since last communication: 0.000s
I20240501 19:56:21.608767 24848 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 145, Last known committed idx: 144, Time since last communication: 0.000s
I20240501 19:56:21.612221 23848 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 143. Preceding OpId from leader: term: 1 index: 146. (index mismatch)
I20240501 19:56:21.626327 25218 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.145: config changed from index 110 to 145, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 145 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.629897 23180 catalog_manager.cc:5484] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 reported cstate change: config changed from index 110 to 145, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 145 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:21.631247 23981 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.145: config changed from index 110 to 145, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 145 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.636713 23452 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.145: config changed from index 110 to 145, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 145 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.645530 24848 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 145, Last known committed idx: 143, Time since last communication: 0.000s
I20240501 19:56:21.654412 24756 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 144. Preceding OpId from leader: term: 1 index: 146. (index mismatch)
I20240501 19:56:21.655476 24848 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 145, Last known committed idx: 144, Time since last communication: 0.000s
I20240501 19:56:21.656803 24756 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.145: config changed from index 110 to 145, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 145 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.664023 23588 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.145: config changed from index 110 to 145, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 145 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
W20240501 19:56:21.766999 23260 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.6:37667, user_credentials={real_user=slave}} blocked reactor thread for 64021.6us
I20240501 19:56:21.773175 25366 ts_tablet_manager.cc:902] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b: Initiating tablet copy from peer 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667)
I20240501 19:56:21.774161 25366 tablet_copy_client.cc:308] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.6:37667
I20240501 19:56:21.775324 23850 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.145: config changed from index 110 to 145, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 145 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.825930 24001 tablet_copy_service.cc:140] P 4f51f982ad284c22aae01c4541a621f1: Received BeginTabletCopySession request for tablet 78b919c4db7745fd87151d8834866e26 from peer 3b3f502cbeb1451087a06f95a3d93f2b ({username='slave'} at 127.20.210.7:58839)
I20240501 19:56:21.826047 24001 tablet_copy_service.cc:161] P 4f51f982ad284c22aae01c4541a621f1: Beginning new tablet copy session on tablet 78b919c4db7745fd87151d8834866e26 from peer 3b3f502cbeb1451087a06f95a3d93f2b at {username='slave'} at 127.20.210.7:58839: session id = 3b3f502cbeb1451087a06f95a3d93f2b-78b919c4db7745fd87151d8834866e26
I20240501 19:56:21.827450 24001 tablet_copy_source_session.cc:205] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:21.828634 25366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78b919c4db7745fd87151d8834866e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:21.830718 25366 tablet_copy_client.cc:783] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:21.830864 25366 tablet_copy_client.cc:647] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:21.840999 25366 tablet_copy_client.cc:523] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:21.842135 25366 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap starting.
I20240501 19:56:21.922861 25362 ts_tablet_manager.cc:902] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b: Initiating tablet copy from peer ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:21.924916 25362 tablet_copy_client.cc:308] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.5:44723
I20240501 19:56:21.926051 23870 tablet_copy_service.cc:140] P ee9bc706484e48fb8ff739a61cabe858: Received BeginTabletCopySession request for tablet 44f96d2d427d43008630eb7bcc39caf9 from peer 3b3f502cbeb1451087a06f95a3d93f2b ({username='slave'} at 127.20.210.7:46877)
I20240501 19:56:21.926143 23870 tablet_copy_service.cc:161] P ee9bc706484e48fb8ff739a61cabe858: Beginning new tablet copy session on tablet 44f96d2d427d43008630eb7bcc39caf9 from peer 3b3f502cbeb1451087a06f95a3d93f2b at {username='slave'} at 127.20.210.7:46877: session id = 3b3f502cbeb1451087a06f95a3d93f2b-44f96d2d427d43008630eb7bcc39caf9
I20240501 19:56:21.927019 23870 tablet_copy_source_session.cc:205] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:21.941295 23719 consensus_queue.cc:237] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 151, Committed index: 151, Last appended: 1.152, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:21.946863 24755 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240501 19:56:21.946954 23586 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240501 19:56:21.948026 24849 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 151, Time since last communication: 0.000s
I20240501 19:56:21.950472 24849 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 151, Time since last communication: 0.000s
I20240501 19:56:21.956482 24849 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.153: config changed from index 145 to 153, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.958909 23171 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 292aa245734c41d9814b8347d543b8fb with cas_config_opid_index 145: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:21.961484 23180 catalog_manager.cc:5484] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 reported cstate change: config changed from index 145 to 153, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:21.965256 23452 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240501 19:56:21.966770 24753 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 145 to 153, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.967595 25362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f96d2d427d43008630eb7bcc39caf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:21.967795 23584 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 145 to 153, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:21.970185 25362 tablet_copy_client.cc:783] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:21.970364 25362 tablet_copy_client.cc:647] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:21.977842 25362 tablet_copy_client.cc:523] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:21.978941 25362 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap starting.
I20240501 19:56:21.991384 23847 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240501 19:56:21.998468 25362 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:21.998713 25362 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap complete.
I20240501 19:56:21.998867 25362 ts_tablet_manager.cc:1367] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20240501 19:56:21.999766 25362 raft_consensus.cc:348] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:21.999874 25362 raft_consensus.cc:729] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3b3f502cbeb1451087a06f95a3d93f2b, State: Initialized, Role: LEARNER
I20240501 19:56:21.999954 25362 consensus_queue.cc:260] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:22.000285 25362 ts_tablet_manager.cc:1398] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:22.003155 23870 tablet_copy_service.cc:342] P ee9bc706484e48fb8ff739a61cabe858: Request end of tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-44f96d2d427d43008630eb7bcc39caf9 received from {username='slave'} at 127.20.210.7:46877
I20240501 19:56:22.003230 23870 tablet_copy_service.cc:433] P ee9bc706484e48fb8ff739a61cabe858: ending tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-44f96d2d427d43008630eb7bcc39caf9 on tablet 44f96d2d427d43008630eb7bcc39caf9 with peer 3b3f502cbeb1451087a06f95a3d93f2b
I20240501 19:56:22.010423 25380 ts_tablet_manager.cc:902] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b: Initiating tablet copy from peer ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:22.011159 25380 tablet_copy_client.cc:308] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.5:44723
I20240501 19:56:22.016270 23870 tablet_copy_service.cc:140] P ee9bc706484e48fb8ff739a61cabe858: Received BeginTabletCopySession request for tablet 107c33af3fbf4edbb3d3b4ba880e4661 from peer 3b3f502cbeb1451087a06f95a3d93f2b ({username='slave'} at 127.20.210.7:46877)
I20240501 19:56:22.016353 23870 tablet_copy_service.cc:161] P ee9bc706484e48fb8ff739a61cabe858: Beginning new tablet copy session on tablet 107c33af3fbf4edbb3d3b4ba880e4661 from peer 3b3f502cbeb1451087a06f95a3d93f2b at {username='slave'} at 127.20.210.7:46877: session id = 3b3f502cbeb1451087a06f95a3d93f2b-107c33af3fbf4edbb3d3b4ba880e4661
I20240501 19:56:22.017267 23870 tablet_copy_source_session.cc:205] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:22.019846 25380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 107c33af3fbf4edbb3d3b4ba880e4661. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:22.036927 25380 tablet_copy_client.cc:783] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:22.037060 25380 tablet_copy_client.cc:647] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:22.037752 23961 tablet_service.cc:1514] Processing DeleteTablet for tablet 292aa245734c41d9814b8347d543b8fb with delete_type TABLET_DATA_TOMBSTONED (TS 4f51f982ad284c22aae01c4541a621f1 not found in new config with opid_index 153) from {username='slave'} at 127.0.0.1:45934
I20240501 19:56:22.039412 25380 tablet_copy_client.cc:523] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:22.040598 25380 tablet_bootstrap.cc:492] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap starting.
W20240501 19:56:22.092326 23655 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.3:37279, user_credentials={real_user=slave}} blocked reactor thread for 126602us
W20240501 19:56:22.092936 23652 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.2:38367, user_credentials={real_user=slave}} blocked reactor thread for 127262us
W20240501 19:56:22.094411 23653 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.7:35019, user_credentials={real_user=slave}} blocked reactor thread for 128611us
I20240501 19:56:22.137792 25366 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=168 overwritten=0 applied=168 ignored=0} inserts{seen=840 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:22.143270 25397 tablet_replica.cc:330] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1: stopping tablet replica
I20240501 19:56:22.143471 25397 raft_consensus.cc:2229] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:22.143707 25397 pending_rounds.cc:62] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1: Trying to abort 1 pending ops.
I20240501 19:56:22.143802 25397 pending_rounds.cc:68] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1: Aborting op as it isn't in flight: id { term: 1 index: 152 } timestamp: 7022974492243005440 op_type: WRITE_OP write_request { tablet_id: "292aa245734c41d9814b8347d543b8fb" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\273\013\345\023\233V8\240\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""993bcee2cb170a65" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7022974492220080128 authz_token { token_data: "\010\212\277\312\261\006\"3\n\005slave\022*\n 25de88db313d414491562faffbac5131\020\001\030\001 \001(\001" signature: "<redacted>""\305XekhAc\177W\333\232\221\241.\272\005\203E\274Y\206us\'\022\360u\210\312l\n4$\211\377\033CW\212gT\2132\0001\266\266mH\037u\302\251 \260r\037\262@\353\357;\314\374" signing_key_seq_num: 0 } } request_id { client_id: "e0a65c6ccb9c4814afd4895b552770e5" seq_no: 446 first_incomplete_seq_no: 446 attempt_no: 0 }
I20240501 19:56:22.145542 25397 raft_consensus.cc:2258] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:22.150596 25366 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap complete.
I20240501 19:56:22.150887 25366 ts_tablet_manager.cc:1367] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent bootstrapping tablet: real 0.309s user 0.139s sys 0.010s
I20240501 19:56:22.151443 25366 raft_consensus.cc:348] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 158 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:22.152163 25366 raft_consensus.cc:729] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3b3f502cbeb1451087a06f95a3d93f2b, State: Initialized, Role: LEARNER
I20240501 19:56:22.152354 25366 consensus_queue.cc:260] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 168, Last appended: 1.168, Last appended by leader: 168, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 158 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:22.152817 25366 ts_tablet_manager.cc:1398] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:22.153407 24001 tablet_copy_service.cc:342] P 4f51f982ad284c22aae01c4541a621f1: Request end of tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-78b919c4db7745fd87151d8834866e26 received from {username='slave'} at 127.20.210.7:58839
I20240501 19:56:22.153496 24001 tablet_copy_service.cc:433] P 4f51f982ad284c22aae01c4541a621f1: ending tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-78b919c4db7745fd87151d8834866e26 on tablet 78b919c4db7745fd87151d8834866e26 with peer 3b3f502cbeb1451087a06f95a3d93f2b
I20240501 19:56:22.171847 25397 ts_tablet_manager.cc:1875] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:22.174778 25397 ts_tablet_manager.cc:1888] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.152
I20240501 19:56:22.178622 25397 log.cc:1196] T 292aa245734c41d9814b8347d543b8fb P 4f51f982ad284c22aae01c4541a621f1: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/wals/292aa245734c41d9814b8347d543b8fb
I20240501 19:56:22.207218 24756 raft_consensus.cc:1203] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.157->[1.158-1.182] Dedup: 1.168->[1.169-1.182]
I20240501 19:56:22.224772 24753 raft_consensus.cc:1203] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240501 19:56:22.244339 24821 raft_consensus.cc:1050] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1: attempting to promote NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b to VOTER
I20240501 19:56:22.244834 24821 consensus_queue.cc:237] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 182, Committed index: 182, Last appended: 1.183, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 184 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:22.247344 23454 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 183. Preceding OpId from leader: term: 1 index: 184. (index mismatch)
I20240501 19:56:22.247515 24755 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 183. Preceding OpId from leader: term: 1 index: 184. (index mismatch)
I20240501 19:56:22.247689 24484 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 184, Last known committed idx: 182, Time since last communication: 0.000s
I20240501 19:56:22.247851 24827 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 184, Last known committed idx: 182, Time since last communication: 0.000s
I20240501 19:56:22.248890 23587 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 183. Preceding OpId from leader: term: 1 index: 184. (index mismatch)
I20240501 19:56:22.249421 24484 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 184, Last known committed idx: 182, Time since last communication: 0.000s
I20240501 19:56:22.262234 24484 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 1 LEADER]: Committing config change with OpId 1.184: config changed from index 158 to 184, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 184 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.266449 24757 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.184: config changed from index 158 to 184, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 184 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.269029 23178 catalog_manager.cc:5484] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b reported cstate change: config changed from index 158 to 184, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4f51f982ad284c22aae01c4541a621f1" committed_config { opid_index: 184 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.273051 23587 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.184: config changed from index 158 to 184, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 184 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.292207 23847 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 184. (index mismatch)
I20240501 19:56:22.292680 24484 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 184, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 19:56:22.296206 23454 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.184: config changed from index 158 to 184, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 184 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.305400 23169 catalog_manager.cc:4830] TS 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667): tablet 292aa245734c41d9814b8347d543b8fb (table rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]) successfully deleted
I20240501 19:56:22.307233 23321 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 175. Preceding OpId from leader: term: 1 index: 184. (index mismatch)
I20240501 19:56:22.322959 25380 tablet_bootstrap.cc:492] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=172 overwritten=0 applied=172 ignored=0} inserts{seen=857 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:22.323256 25380 tablet_bootstrap.cc:492] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap complete.
I20240501 19:56:22.323518 25380 ts_tablet_manager.cc:1367] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent bootstrapping tablet: real 0.283s user 0.118s sys 0.020s
I20240501 19:56:22.324008 25380 raft_consensus.cc:348] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:22.324213 25380 raft_consensus.cc:729] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3b3f502cbeb1451087a06f95a3d93f2b, State: Initialized, Role: LEARNER
I20240501 19:56:22.324374 25380 consensus_queue.cc:260] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 172, Last appended: 1.172, Last appended by leader: 172, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:22.324956 25380 ts_tablet_manager.cc:1398] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:22.325551 24484 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 184, Last known committed idx: 175, Time since last communication: 0.000s
I20240501 19:56:22.326597 23870 tablet_copy_service.cc:342] P ee9bc706484e48fb8ff739a61cabe858: Request end of tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-107c33af3fbf4edbb3d3b4ba880e4661 received from {username='slave'} at 127.20.210.7:46877
I20240501 19:56:22.326668 23870 tablet_copy_service.cc:433] P ee9bc706484e48fb8ff739a61cabe858: ending tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-107c33af3fbf4edbb3d3b4ba880e4661 on tablet 107c33af3fbf4edbb3d3b4ba880e4661 with peer 3b3f502cbeb1451087a06f95a3d93f2b
I20240501 19:56:22.331396 23850 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.184: config changed from index 158 to 184, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 184 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.336882 23321 raft_consensus.cc:1203] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.175->[1.176-1.185] Dedup: 1.181->[1.182-1.185]
I20240501 19:56:22.339855 23321 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.184: config changed from index 158 to 184, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 184 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
W20240501 19:56:22.347435 23652 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.2:38367, user_credentials={real_user=slave}} blocked reactor thread for 253921us
I20240501 19:56:22.349159 24849 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 151, Time since last communication: 0.000s
I20240501 19:56:22.354712 23455 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 145 to 153, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.357048 25073 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 151, Time since last communication: 0.000s
I20240501 19:56:22.363430 23979 consensus_queue.cc:237] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 185, Committed index: 185, Last appended: 1.186, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 187 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:22.365244 24755 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 186. Preceding OpId from leader: term: 1 index: 187. (index mismatch)
I20240501 19:56:22.366294 24484 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 187, Last known committed idx: 185, Time since last communication: 0.000s
I20240501 19:56:22.366637 23587 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 185. Preceding OpId from leader: term: 1 index: 187. (index mismatch)
I20240501 19:56:22.367002 24484 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 187, Last known committed idx: 185, Time since last communication: 0.000s
I20240501 19:56:22.369310 23849 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 185. Preceding OpId from leader: term: 1 index: 187. (index mismatch)
I20240501 19:56:22.370409 23584 raft_consensus.cc:1203] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.185->[1.186-1.186] Dedup: 1.186->[]
I20240501 19:56:22.371096 25041 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 1 LEADER]: Committing config change with OpId 1.187: config changed from index 184 to 187, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 187 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.371246 24755 raft_consensus.cc:1203] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.151->[1.152-1.177] Dedup: 1.172->[1.173-1.177]
I20240501 19:56:22.371491 25325 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 187, Last known committed idx: 185, Time since last communication: 0.000s
I20240501 19:56:22.373025 23849 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.187: config changed from index 184 to 187, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 187 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.374722 24753 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.187: config changed from index 184 to 187, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 187 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.376485 23180 catalog_manager.cc:5484] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 reported cstate change: config changed from index 184 to 187, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "4f51f982ad284c22aae01c4541a621f1" committed_config { opid_index: 187 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:22.385509 23435 tablet_service.cc:1514] Processing DeleteTablet for tablet 78b919c4db7745fd87151d8834866e26 with delete_type TABLET_DATA_TOMBSTONED (TS 0496e037179e4b3ca72aead781c25d5a not found in new config with opid_index 187) from {username='slave'} at 127.0.0.1:49870
I20240501 19:56:22.386293 23324 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 186. Preceding OpId from leader: term: 1 index: 187. (index mismatch)
I20240501 19:56:22.387018 24484 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 187, Last known committed idx: 185, Time since last communication: 0.000s
I20240501 19:56:22.387265 23169 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 78b919c4db7745fd87151d8834866e26 with cas_config_opid_index 184: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:22.389148 23582 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.187: config changed from index 184 to 187, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 187 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.397105 23848 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 145 to 153, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.415424 23324 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.187: config changed from index 184 to 187, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 187 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.425052 25405 tablet_replica.cc:330] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a: stopping tablet replica
I20240501 19:56:22.425246 25405 raft_consensus.cc:2229] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:22.425395 25405 pending_rounds.cc:62] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a: Trying to abort 1 pending ops.
I20240501 19:56:22.425518 25405 pending_rounds.cc:68] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a: Aborting op as it isn't in flight: id { term: 1 index: 186 } timestamp: 7022974494127149056 op_type: WRITE_OP write_request { tablet_id: "78b919c4db7745fd87151d8834866e26" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\215\006\237m\350uz\022\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""5b2ee5b247d42e1e" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7022974494073151488 authz_token { token_data: "\010\212\277\312\261\006\"3\n\005slave\022*\n ffcc6e469144432faca7b1c39980bdb7\020\001\030\001 \001(\001" signature: "<redacted>""\315#\317h\234>\237\270\220z\2258U\265Ec\211\374vF\350\361\245\233U\323\276+\212\363Og\310\302{\250\213C\301?\314H`M\320i\332+\236\025\275S.!\245b\225\331\025\260*h\275\356" signing_key_seq_num: 0 } } request_id { client_id: "017df3424f7a4b41b7d7928bfb2dbb08" seq_no: 517 first_incomplete_seq_no: 517 attempt_no: 0 }
I20240501 19:56:22.449970 25405 raft_consensus.cc:2258] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:22.472323 25405 ts_tablet_manager.cc:1875] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:22.486366 25405 ts_tablet_manager.cc:1888] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.186
I20240501 19:56:22.486506 25405 log.cc:1196] T 78b919c4db7745fd87151d8834866e26 P 0496e037179e4b3ca72aead781c25d5a: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/78b919c4db7745fd87151d8834866e26
I20240501 19:56:22.494104 23168 catalog_manager.cc:4830] TS 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367): tablet 78b919c4db7745fd87151d8834866e26 (table rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]) successfully deleted
I20240501 19:56:22.569882 24837 raft_consensus.cc:1050] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858: attempting to promote NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b to VOTER
I20240501 19:56:22.570286 24837 consensus_queue.cc:237] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 180, Committed index: 180, Last appended: 1.180, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:22.572324 23586 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 19:56:22.574942 25288 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 19:56:22.576179 24755 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 19:56:22.576515 25288 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 19:56:22.582057 23452 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 180. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 19:56:22.590629 25372 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 180, Time since last communication: 0.000s
I20240501 19:56:22.599877 23718 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 19:56:22.619478 25288 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 179, Time since last communication: 0.000s
I20240501 19:56:22.627514 23326 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 179. Preceding OpId from leader: term: 1 index: 181. (index mismatch)
I20240501 19:56:22.654909 25000 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Committing config change with OpId 1.181: config changed from index 152 to 181, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.657006 23584 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 152 to 181, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.659708 23182 catalog_manager.cc:5484] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 reported cstate change: config changed from index 152 to 181, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.664963 23717 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 152 to 181, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.674855 25372 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 181, Last known committed idx: 178, Time since last communication: 0.000s
I20240501 19:56:22.681671 24755 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 152 to 181, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.700399 23321 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 152 to 181, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
W20240501 19:56:22.774034 23391 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.6:37667, user_credentials={real_user=slave}} blocked reactor thread for 59290.9us
W20240501 19:56:22.775560 23785 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.1:35481, user_credentials={real_user=slave}} blocked reactor thread for 56922.4us
I20240501 19:56:22.776611 23849 consensus_queue.cc:237] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 181, Committed index: 181, Last appended: 1.182, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 183 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:22.779727 23717 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 182. Preceding OpId from leader: term: 1 index: 183. (index mismatch)
I20240501 19:56:22.780633 25372 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 183, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 19:56:22.787992 24753 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 182. Preceding OpId from leader: term: 1 index: 183. (index mismatch)
I20240501 19:56:22.839488 23452 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.181: config changed from index 152 to 181, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 181 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.852344 23321 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 182. Preceding OpId from leader: term: 1 index: 183. (index mismatch)
I20240501 19:56:22.853929 23456 raft_consensus.cc:1261] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 182. Preceding OpId from leader: term: 1 index: 183. (index mismatch)
I20240501 19:56:22.854913 25414 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 183, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 19:56:22.855279 25372 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 183, Last known committed idx: 181, Time since last communication: 0.000s
I20240501 19:56:22.856674 25372 consensus_queue.cc:1035] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 183, Last known committed idx: 181, Time since last communication: 0.000s
W20240501 19:56:22.856719 23784 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.7:35019, user_credentials={real_user=slave}} blocked reactor thread for 67310.1us
I20240501 19:56:22.860723 25288 raft_consensus.cc:1050] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858: attempting to promote NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b to VOTER
I20240501 19:56:22.861356 25288 consensus_queue.cc:237] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:22.863662 23717 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 19:56:22.864553 25414 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 19:56:22.866824 24753 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 19:56:22.867105 23979 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 19:56:22.867344 25372 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Committing config change with OpId 1.183: config changed from index 181 to 183, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 183 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.869206 23717 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.183: config changed from index 181 to 183, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 183 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.870735 23168 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 107c33af3fbf4edbb3d3b4ba880e4661 with cas_config_opid_index 181: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:22.874646 23452 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 19:56:22.876123 23178 catalog_manager.cc:5484] T 107c33af3fbf4edbb3d3b4ba880e4661 P 6e47806645ae47278e35600cc67d5700 reported cstate change: config changed from index 181 to 183, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: 183 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.877344 25156 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 19:56:22.878635 23568 tablet_service.cc:1514] Processing DeleteTablet for tablet 107c33af3fbf4edbb3d3b4ba880e4661 with delete_type TABLET_DATA_TOMBSTONED (TS e8ddd74cfd954c7a8c2d284c335d7826 not found in new config with opid_index 183) from {username='slave'} at 127.0.0.1:46492
I20240501 19:56:22.880646 23326 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 19:56:22.884361 23324 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.183: config changed from index 181 to 183, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 183 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.887755 25414 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 19:56:22.891729 24756 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.183: config changed from index 181 to 183, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 183 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.893460 25156 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 19:56:22.893685 25338 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240501 19:56:22.907853 23452 raft_consensus.cc:2932] T 107c33af3fbf4edbb3d3b4ba880e4661 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.183: config changed from index 181 to 183, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 183 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:22.942000 25425 tablet_replica.cc:330] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826: stopping tablet replica
I20240501 19:56:22.942183 25425 raft_consensus.cc:2229] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:22.942382 25425 pending_rounds.cc:62] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826: Trying to abort 1 pending ops.
I20240501 19:56:22.942512 25425 pending_rounds.cc:68] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826: Aborting op as it isn't in flight: id { term: 1 index: 182 } timestamp: 7022974495472295936 op_type: WRITE_OP write_request { tablet_id: "107c33af3fbf4edbb3d3b4ba880e4661" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\305l+XA\006\315\037\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""18f93bb2257c5fda" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7022974495331807232 authz_token { token_data: "\010\212\277\312\261\006\"3\n\005slave\022*\n ffcc6e469144432faca7b1c39980bdb7\020\001\030\001 \001(\001" signature: "<redacted>""\315#\317h\234>\237\270\220z\2258U\265Ec\211\374vF\350\361\245\233U\323\276+\212\363Og\310\302{\250\213C\301?\314H`M\320i\332+\236\025\275S.!\245b\225\331\025\260*h\275\356" signing_key_seq_num: 0 } } request_id { client_id: "017df3424f7a4b41b7d7928bfb2dbb08" seq_no: 536 first_incomplete_seq_no: 536 attempt_no: 0 }
I20240501 19:56:22.942970 25425 raft_consensus.cc:2258] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:22.943801 25425 ts_tablet_manager.cc:1875] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:22.997031 25288 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:23.008713 23717 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:23.009433 23178 catalog_manager.cc:5484] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index 2 to 3, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:23.013403 23975 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:23.015612 24755 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:23.016090 23452 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:23.020350 23324 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:23.031373 23849 consensus_queue.cc:237] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:23.037148 24756 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:23.037914 23981 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:23.038499 25288 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:23.040794 25414 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:23.045720 23452 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:23.048586 25338 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:23.050719 23717 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:23.051376 23980 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:23.052004 23178 catalog_manager.cc:5484] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index 3 to 4, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:23.054033 23168 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 44f96d2d427d43008630eb7bcc39caf9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:23.056329 23305 tablet_service.cc:1514] Processing DeleteTablet for tablet 44f96d2d427d43008630eb7bcc39caf9 with delete_type TABLET_DATA_TOMBSTONED (TS ad9452840e4e4184a640fb63dab754fd not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55042
I20240501 19:56:23.057178 25338 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:23.057488 25338 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:23.060856 25430 tablet_replica.cc:330] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd: stopping tablet replica
I20240501 19:56:23.061017 25430 raft_consensus.cc:2229] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:23.061195 25430 raft_consensus.cc:2258] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:23.066030 25425 ts_tablet_manager.cc:1888] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.182
I20240501 19:56:23.066510 23454 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:23.073599 24755 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:23.076709 25425 log.cc:1196] T 107c33af3fbf4edbb3d3b4ba880e4661 P e8ddd74cfd954c7a8c2d284c335d7826: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/107c33af3fbf4edbb3d3b4ba880e4661
I20240501 19:56:23.086324 23171 catalog_manager.cc:4830] TS e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279): tablet 107c33af3fbf4edbb3d3b4ba880e4661 (table rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]) successfully deleted
I20240501 19:56:23.110422 23322 consensus_queue.cc:237] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 87, Committed index: 87, Last appended: 1.88, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
W20240501 19:56:23.113159 23260 consensus_peers.cc:485] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd -> Peer 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019): Couldn't send request to peer 3b3f502cbeb1451087a06f95a3d93f2b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a4f98ae45e914fa3981237af149b0666. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:23.113363 23587 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 88. Preceding OpId from leader: term: 1 index: 89. (index mismatch)
I20240501 19:56:23.113788 25023 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 89, Last known committed idx: 87, Time since last communication: 0.000s
I20240501 19:56:23.117064 23849 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 88. Preceding OpId from leader: term: 1 index: 89. (index mismatch)
I20240501 19:56:23.118801 23455 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 88. Preceding OpId from leader: term: 1 index: 89. (index mismatch)
I20240501 19:56:23.124502 25023 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 89, Last known committed idx: 87, Time since last communication: 0.000s
I20240501 19:56:23.125422 25023 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 89, Last known committed idx: 87, Time since last communication: 0.000s
I20240501 19:56:23.127688 23718 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:23.136590 25430 ts_tablet_manager.cc:1875] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:23.140046 24519 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.89: config changed from index -1 to 89, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:23.141206 23454 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.89: config changed from index -1 to 89, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:23.141981 23849 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.89: config changed from index -1 to 89, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:23.145761 23178 catalog_manager.cc:5484] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index -1 to 89, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:23.146188 25430 ts_tablet_manager.cc:1888] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240501 19:56:23.146996 25430 log.cc:1196] T 44f96d2d427d43008630eb7bcc39caf9 P ad9452840e4e4184a640fb63dab754fd: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/44f96d2d427d43008630eb7bcc39caf9
I20240501 19:56:23.148329 23170 catalog_manager.cc:4830] TS ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481): tablet 44f96d2d427d43008630eb7bcc39caf9 (table TestTable [id=e130370886f346e289c105d0836dc0da]) successfully deleted
I20240501 19:56:23.149780 23588 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.89: config changed from index -1 to 89, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
I20240501 19:56:23.155757 23718 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 87. Preceding OpId from leader: term: 1 index: 89. (index mismatch)
I20240501 19:56:23.183140 25439 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 89, Last known committed idx: 87, Time since last communication: 0.000s
I20240501 19:56:23.250247 23717 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.89: config changed from index -1 to 89, NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) added. New config: { opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } } }
W20240501 19:56:23.390718 23393 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.4:44657, user_credentials={real_user=slave}} blocked reactor thread for 62259.3us
I20240501 19:56:23.777596 25464 ts_tablet_manager.cc:902] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b: Initiating tablet copy from peer ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481)
I20240501 19:56:23.778328 25464 tablet_copy_client.cc:308] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:35481
I20240501 19:56:23.778884 23346 tablet_copy_service.cc:140] P ad9452840e4e4184a640fb63dab754fd: Received BeginTabletCopySession request for tablet a4f98ae45e914fa3981237af149b0666 from peer 3b3f502cbeb1451087a06f95a3d93f2b ({username='slave'} at 127.20.210.7:35207)
I20240501 19:56:23.778973 23346 tablet_copy_service.cc:161] P ad9452840e4e4184a640fb63dab754fd: Beginning new tablet copy session on tablet a4f98ae45e914fa3981237af149b0666 from peer 3b3f502cbeb1451087a06f95a3d93f2b at {username='slave'} at 127.20.210.7:35207: session id = 3b3f502cbeb1451087a06f95a3d93f2b-a4f98ae45e914fa3981237af149b0666
I20240501 19:56:23.779807 23346 tablet_copy_source_session.cc:205] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:23.866740 25464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4f98ae45e914fa3981237af149b0666. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:23.943861 25464 tablet_copy_client.cc:783] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:23.962323 25464 tablet_copy_client.cc:647] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Starting download of 1 WAL segments...
W20240501 19:56:23.989322 23391 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.6:37667, user_credentials={real_user=slave}} blocked reactor thread for 56884.2us
I20240501 19:56:24.000229 25464 tablet_copy_client.cc:523] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:24.002494 25464 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap starting.
I20240501 19:56:24.636236 25464 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=99 overwritten=0 applied=99 ignored=0} inserts{seen=784 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:24.636596 25464 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b: Bootstrap complete.
I20240501 19:56:24.636785 25464 ts_tablet_manager.cc:1367] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent bootstrapping tablet: real 0.634s user 0.082s sys 0.008s
I20240501 19:56:24.637219 25464 raft_consensus.cc:348] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:24.637346 25464 raft_consensus.cc:729] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3b3f502cbeb1451087a06f95a3d93f2b, State: Initialized, Role: LEARNER
I20240501 19:56:24.637449 25464 consensus_queue.cc:260] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 99, Last appended: 1.99, Last appended by leader: 99, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 89 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: true } }
I20240501 19:56:24.637766 25464 ts_tablet_manager.cc:1398] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 19:56:24.638172 23346 tablet_copy_service.cc:342] P ad9452840e4e4184a640fb63dab754fd: Request end of tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-a4f98ae45e914fa3981237af149b0666 received from {username='slave'} at 127.20.210.7:35207
I20240501 19:56:24.638259 23346 tablet_copy_service.cc:433] P ad9452840e4e4184a640fb63dab754fd: ending tablet copy session 3b3f502cbeb1451087a06f95a3d93f2b-a4f98ae45e914fa3981237af149b0666 on tablet a4f98ae45e914fa3981237af149b0666 with peer 3b3f502cbeb1451087a06f95a3d93f2b
I20240501 19:56:24.860931 24751 raft_consensus.cc:1203] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.88->[1.89-1.114] Dedup: 1.99->[1.100-1.114]
I20240501 19:56:24.931017 24395 raft_consensus.cc:1050] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd: attempting to promote NON_VOTER 3b3f502cbeb1451087a06f95a3d93f2b to VOTER
I20240501 19:56:24.931715 24395 consensus_queue.cc:237] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 115, Committed index: 115, Last appended: 1.116, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:24.933790 23584 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 116. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240501 19:56:24.934286 25023 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 115, Time since last communication: 0.000s
I20240501 19:56:24.946936 23846 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 116. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240501 19:56:24.948454 23719 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 115. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240501 19:56:24.955957 24756 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 LEARNER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 116. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240501 19:56:24.958570 25439 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 115, Time since last communication: 0.000s
I20240501 19:56:24.958710 25023 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 114, Time since last communication: 0.000s
I20240501 19:56:24.958884 25439 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 115, Time since last communication: 0.000s
I20240501 19:56:24.972654 23452 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 116. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240501 19:56:24.977301 24519 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 115, Time since last communication: 0.000s
I20240501 19:56:24.985778 24519 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.117: config changed from index 89 to 117, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:24.991843 23586 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 89 to 117, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:24.994421 23178 catalog_manager.cc:5484] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 reported cstate change: config changed from index 89 to 117, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:24.998741 23849 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 89 to 117, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:24.999167 24756 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 89 to 117, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:25.008316 23719 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 89 to 117, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:25.047273 23452 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 89 to 117, 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:25.102744 23322 consensus_queue.cc:237] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 119, Committed index: 119, Last appended: 1.119, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } }
I20240501 19:56:25.105772 23584 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 119. Preceding OpId from leader: term: 1 index: 120. (index mismatch)
I20240501 19:56:25.107681 23718 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 119. Preceding OpId from leader: term: 1 index: 120. (index mismatch)
I20240501 19:56:25.108004 25023 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 120, Last known committed idx: 119, Time since last communication: 0.000s
I20240501 19:56:25.110533 25023 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 120, Last known committed idx: 119, Time since last communication: 0.000s
I20240501 19:56:25.111985 24756 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 119. Preceding OpId from leader: term: 1 index: 120. (index mismatch)
I20240501 19:56:25.113350 25023 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 120, Last known committed idx: 119, Time since last communication: 0.000s
I20240501 19:56:25.117573 24395 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.120: config changed from index 117 to 120, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:25.119108 23847 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 119. Preceding OpId from leader: term: 1 index: 120. (index mismatch)
I20240501 19:56:25.120612 23170 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a4f98ae45e914fa3981237af149b0666 with cas_config_opid_index 117: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:25.121882 23586 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.120: config changed from index 117 to 120, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:25.123955 23179 catalog_manager.cc:5484] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index 117 to 120, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:25.129014 23435 tablet_service.cc:1514] Processing DeleteTablet for tablet a4f98ae45e914fa3981237af149b0666 with delete_type TABLET_DATA_TOMBSTONED (TS 0496e037179e4b3ca72aead781c25d5a not found in new config with opid_index 120) from {username='slave'} at 127.0.0.1:49870
I20240501 19:56:25.129972 24751 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.120: config changed from index 117 to 120, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:25.137138 25511 tablet_replica.cc:330] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a: stopping tablet replica
I20240501 19:56:25.137316 25511 raft_consensus.cc:2229] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:25.138033 25511 raft_consensus.cc:2258] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:25.139039 25511 ts_tablet_manager.cc:1875] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:25.140973 24395 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 120, Last known committed idx: 119, Time since last communication: 0.000s
I20240501 19:56:25.145542 23716 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.120: config changed from index 117 to 120, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:25.154011 25511 ts_tablet_manager.cc:1888] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.119
I20240501 19:56:25.154093 25511 log.cc:1196] T a4f98ae45e914fa3981237af149b0666 P 0496e037179e4b3ca72aead781c25d5a: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/a4f98ae45e914fa3981237af149b0666
I20240501 19:56:25.167779 23850 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.120: config changed from index 117 to 120, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } }
I20240501 19:56:25.169728 23168 catalog_manager.cc:4830] TS 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367): tablet a4f98ae45e914fa3981237af149b0666 (table rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]) successfully deleted
I20240501 19:56:25.864895 23961 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:25.868203 23305 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:25.869233 24737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:25.884377 23568 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:25.900068 23435 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:25.931100 23830 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
W20240501 19:56:25.944725 23653 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.7:35019, user_credentials={real_user=slave}} blocked reactor thread for 55740.1us
W20240501 19:56:25.944774 23652 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.5:44723, user_credentials={real_user=slave}} blocked reactor thread for 48516.6us
W20240501 19:56:25.944890 23655 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.3:37279, user_credentials={real_user=slave}} blocked reactor thread for 43796.5us
I20240501 19:56:25.971459 23698 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
W20240501 19:56:26.603374 23652 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.5:44723, user_credentials={real_user=slave}} blocked reactor thread for 72969.7us
I20240501 19:56:27.046741 23435 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:27.109807 23305 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:27.118970 23698 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:27.120030 24737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:27.131210 23568 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:27.150628 23830 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:27.189625 23961 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20240501 19:56:27.696429 23652 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.2:38367, user_credentials={real_user=slave}} blocked reactor thread for 45586.1us
W20240501 19:56:27.800355 23259 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.5:44723, user_credentials={real_user=slave}} blocked reactor thread for 85282.8us
I20240501 19:56:28.853099 23961 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:28.888149 24737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:28.894490 23305 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:28.908260 23698 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:28.910116 23830 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:28.990747 23568 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20240501 19:56:29.005618 23391 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.20.210.7:35019, user_credentials={real_user=slave}} blocked reactor thread for 46382.1us
I20240501 19:56:29.032938 23435 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:30.290015 23435 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:30.362803 24737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:30.371935 23698 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:30.380461 23568 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:30.458035 23305 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:30.579877 23830 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:30.582978 23961 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:31.087091 23435 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:31.089818 24737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:31.090108 23961 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:31.097003 23568 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:31.104444 23305 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:31.110105 23698 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:31.120424 23830 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
bb28524133a341a69c1f36bc25de2d8d | 127.20.210.62:37935 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.20.210.20:35945 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
0496e037179e4b3ca72aead781c25d5a | 127.20.210.2:38367 | HEALTHY | <none> | 3 | 0
3b3f502cbeb1451087a06f95a3d93f2b | 127.20.210.7:35019 | HEALTHY | <none> | 0 | 0
4f51f982ad284c22aae01c4541a621f1 | 127.20.210.6:37667 | HEALTHY | <none> | 1 | 0
6e47806645ae47278e35600cc67d5700 | 127.20.210.4:44657 | HEALTHY | <none> | 4 | 0
ad9452840e4e4184a640fb63dab754fd | 127.20.210.1:35481 | HEALTHY | <none> | 3 | 0
e8ddd74cfd954c7a8c2d284c335d7826 | 127.20.210.3:37279 | HEALTHY | <none> | 1 | 0
ee9bc706484e48fb8ff739a61cabe858 | 127.20.210.5:44723 | HEALTHY | <none> | 4 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 7
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 7 server(s) checked
local_ip_for_outbound_sockets | 127.20.210.1 | experimental | 127.20.210.1:35481
local_ip_for_outbound_sockets | 127.20.210.2 | experimental | 127.20.210.2:38367
local_ip_for_outbound_sockets | 127.20.210.3 | experimental | 127.20.210.3:37279
local_ip_for_outbound_sockets | 127.20.210.4 | experimental | 127.20.210.4:44657
local_ip_for_outbound_sockets | 127.20.210.5 | experimental | 127.20.210.5:44723
local_ip_for_outbound_sockets | 127.20.210.6 | experimental | 127.20.210.6:37667
local_ip_for_outbound_sockets | 127.20.210.7 | experimental | 127.20.210.7:35019
never_fsync | true | unsafe,advanced | all 7 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 7 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 7 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 7 server(s) checked
server_dump_info_format | pb | hidden | all 7 server(s) checked
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.20.210.1:35481
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.20.210.2:38367
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.20.210.3:37279
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.20.210.4:44657
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.20.210.5:44723
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.20.210.6:37667
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.20.210.7:35019
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.20.210.20:35945 | all 7 server(s) checked
time_source | builtin | all 7 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 8 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 5 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 11
First Quartile | 11
Median | 11
Third Quartile | 12
Maximum | 12
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 7
Tables | 6
Tablets | 16
Replicas | 80
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240501 19:56:31.144742 21320 log_verifier.cc:126] Checking tablet 107c33af3fbf4edbb3d3b4ba880e4661
I20240501 19:56:31.298556 21320 log_verifier.cc:177] Verified matching terms for 359 ops in tablet 107c33af3fbf4edbb3d3b4ba880e4661
I20240501 19:56:31.298782 21320 log_verifier.cc:126] Checking tablet 1c9623f516f84445ade4bf06eb05673f
I20240501 19:56:31.415612 21320 log_verifier.cc:177] Verified matching terms for 263 ops in tablet 1c9623f516f84445ade4bf06eb05673f
I20240501 19:56:31.415756 21320 log_verifier.cc:126] Checking tablet 292aa245734c41d9814b8347d543b8fb
I20240501 19:56:31.577412 21320 log_verifier.cc:177] Verified matching terms for 340 ops in tablet 292aa245734c41d9814b8347d543b8fb
I20240501 19:56:31.577600 21320 log_verifier.cc:126] Checking tablet 44f96d2d427d43008630eb7bcc39caf9
I20240501 19:56:31.615483 21320 log_verifier.cc:177] Verified matching terms for 4 ops in tablet 44f96d2d427d43008630eb7bcc39caf9
I20240501 19:56:31.615546 21320 log_verifier.cc:126] Checking tablet 5898d010acd94c35a73a6e5c28f8d313
I20240501 19:56:31.740177 21320 log_verifier.cc:177] Verified matching terms for 253 ops in tablet 5898d010acd94c35a73a6e5c28f8d313
I20240501 19:56:31.740334 21320 log_verifier.cc:126] Checking tablet 6f549fed054d433381ac6d38d64c74bb
I20240501 19:56:31.867383 21320 log_verifier.cc:177] Verified matching terms for 260 ops in tablet 6f549fed054d433381ac6d38d64c74bb
I20240501 19:56:31.867529 21320 log_verifier.cc:126] Checking tablet 78b919c4db7745fd87151d8834866e26
I20240501 19:56:32.026918 21320 log_verifier.cc:177] Verified matching terms for 394 ops in tablet 78b919c4db7745fd87151d8834866e26
I20240501 19:56:32.027135 21320 log_verifier.cc:126] Checking tablet 87ebb3dfc00d4744960bde9a07c59e23
I20240501 19:56:32.184697 21320 log_verifier.cc:177] Verified matching terms for 364 ops in tablet 87ebb3dfc00d4744960bde9a07c59e23
I20240501 19:56:32.184907 21320 log_verifier.cc:126] Checking tablet 9f0625a730b0408b963d94f24e25fed7
I20240501 19:56:32.302130 21320 log_verifier.cc:177] Verified matching terms for 258 ops in tablet 9f0625a730b0408b963d94f24e25fed7
I20240501 19:56:32.302314 21320 log_verifier.cc:126] Checking tablet a4ef2fdbb40e48b18025136adb39defc
I20240501 19:56:32.419261 21320 log_verifier.cc:177] Verified matching terms for 236 ops in tablet a4ef2fdbb40e48b18025136adb39defc
I20240501 19:56:32.419441 21320 log_verifier.cc:126] Checking tablet a4f98ae45e914fa3981237af149b0666
I20240501 19:56:32.524278 21320 log_verifier.cc:177] Verified matching terms for 213 ops in tablet a4f98ae45e914fa3981237af149b0666
I20240501 19:56:32.524485 21320 log_verifier.cc:126] Checking tablet af8e2ce72fb24a18b54b0d2e49383b41
I20240501 19:56:32.678222 21320 log_verifier.cc:177] Verified matching terms for 366 ops in tablet af8e2ce72fb24a18b54b0d2e49383b41
I20240501 19:56:32.678490 21320 log_verifier.cc:126] Checking tablet b1a0cebcdad24e34abc11eef9d4d891d
I20240501 19:56:32.795651 21320 log_verifier.cc:177] Verified matching terms for 253 ops in tablet b1a0cebcdad24e34abc11eef9d4d891d
I20240501 19:56:32.795815 21320 log_verifier.cc:126] Checking tablet b2eea4673ba94b0fa5a2e0f099532777
I20240501 19:56:32.960951 21320 log_verifier.cc:177] Verified matching terms for 367 ops in tablet b2eea4673ba94b0fa5a2e0f099532777
I20240501 19:56:32.961145 21320 log_verifier.cc:126] Checking tablet f53fd7f6fb694eb89eb611497e064ab5
I20240501 19:56:33.075320 21320 log_verifier.cc:177] Verified matching terms for 240 ops in tablet f53fd7f6fb694eb89eb611497e064ab5
I20240501 19:56:33.075497 21320 log_verifier.cc:126] Checking tablet f701e0b045d540698ba3879954d8b171
I20240501 19:56:33.208093 21320 log_verifier.cc:177] Verified matching terms for 258 ops in tablet f701e0b045d540698ba3879954d8b171
I20240501 19:56:33.209826 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.8:0
--local_ip_for_outbound_sockets=127.20.210.8
--webserver_interface=127.20.210.8
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--builtin_ntp_servers=127.20.210.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 19:56:33.312131 25802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:33.312286 25802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:33.312332 25802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:33.315318 25802 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:33.315371 25802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:33.315449 25802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.8
I20240501 19:56:33.318591 25802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.8:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.20.210.8
--webserver_port=0
--tserver_master_addrs=127.20.210.62:37935
--never_fsync=true
--heap_profile_path=/tmp/kudu.25802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.8
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:33.319226 25802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:33.320118 25802 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:33.327811 25807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:33.328574 25808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:33.329275 25810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:33.329948 25802 server_base.cc:1034] running on GCE node
I20240501 19:56:33.330237 25802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:33.330706 25802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:33.331837 25802 hybrid_clock.cc:648] HybridClock initialized: now 1714593393331818 us; error 31 us; skew 500 ppm
I20240501 19:56:33.338696 25802 webserver.cc:460] Webserver started at http://127.20.210.8:33653/ using document root <none> and password file <none>
I20240501 19:56:33.339164 25802 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:33.339216 25802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:33.339383 25802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:33.342818 25802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "d6f0e2c51e3d4598b54766125f1d811f"
format_stamp: "Formatted at 2024-05-01 19:56:33 on dist-test-slave-vgh4"
I20240501 19:56:33.343298 25802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "d6f0e2c51e3d4598b54766125f1d811f"
format_stamp: "Formatted at 2024-05-01 19:56:33 on dist-test-slave-vgh4"
I20240501 19:56:33.346567 25802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 19:56:33.348387 25816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:33.349265 25802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:33.349360 25802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "d6f0e2c51e3d4598b54766125f1d811f"
format_stamp: "Formatted at 2024-05-01 19:56:33 on dist-test-slave-vgh4"
I20240501 19:56:33.349426 25802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:33.368479 25802 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:33.368949 25802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:33.369082 25802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:33.369498 25802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:33.370199 25802 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:33.370266 25802 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:33.370323 25802 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:33.370345 25802 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:33.379205 25802 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.8:35395
I20240501 19:56:33.379235 25929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.8:35395 every 8 connection(s)
I20240501 19:56:33.380028 25802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 19:56:33.385390 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 25802
I20240501 19:56:33.385481 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240501 19:56:33.387912 25930 heartbeater.cc:347] Connected to a master server at 127.20.210.62:37935
I20240501 19:56:33.388075 25930 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:33.388478 25930 heartbeater.cc:511] Master 127.20.210.62:37935 requested a full tablet report, sending...
I20240501 19:56:33.389297 23180 ts_manager.cc:194] Registered new tserver with Master: d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8:35395)
I20240501 19:56:33.389925 23180 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.8:41183
I20240501 19:56:33.560868 23435 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:33.564433 24737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:33.564752 23305 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:33.571842 23830 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:33.572567 25864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:33.572875 23698 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:33.597129 23568 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:33.600939 23961 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:33.675036 23698 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:33.675915 23435 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:33.678287 23305 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:33.678393 24737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:33.685379 23830 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:33.697333 23961 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:33.711356 25864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:33.719025 23568 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:33.747433 23716 consensus_queue.cc:237] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 260, Committed index: 260, Last appended: 1.260, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:33.749163 23849 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 260. Preceding OpId from leader: term: 1 index: 261. (index mismatch)
I20240501 19:56:33.749220 23326 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 260. Preceding OpId from leader: term: 1 index: 261. (index mismatch)
I20240501 19:56:33.749526 23453 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 260. Preceding OpId from leader: term: 1 index: 261. (index mismatch)
I20240501 19:56:33.749543 26027 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 261, Last known committed idx: 260, Time since last communication: 0.000s
I20240501 19:56:33.749931 23581 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 260. Preceding OpId from leader: term: 1 index: 261. (index mismatch)
I20240501 19:56:33.750020 26027 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 261, Last known committed idx: 260, Time since last communication: 0.000s
I20240501 19:56:33.758754 26027 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 261, Last known committed idx: 260, Time since last communication: 0.000s
I20240501 19:56:33.759071 26027 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 261, Last known committed idx: 260, Time since last communication: 0.000s
I20240501 19:56:33.760454 26025 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.261: config changed from index 80 to 261, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.761533 23849 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.261: config changed from index 80 to 261, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
W20240501 19:56:33.762020 23653 consensus_peers.cc:485] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 -> Peer d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8:35395): Couldn't send request to peer d6f0e2c51e3d4598b54766125f1d811f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f549fed054d433381ac6d38d64c74bb. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:33.762346 23581 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.261: config changed from index 80 to 261, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.763119 23326 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.261: config changed from index 80 to 261, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.764084 23453 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.261: config changed from index 80 to 261, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.764364 23184 catalog_manager.cc:5484] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index 80 to 261, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.775483 23326 consensus_queue.cc:237] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 263, Committed index: 263, Last appended: 1.263, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:33.777129 23849 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 263. Preceding OpId from leader: term: 1 index: 264. (index mismatch)
I20240501 19:56:33.777134 23581 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 263. Preceding OpId from leader: term: 1 index: 264. (index mismatch)
I20240501 19:56:33.777489 26044 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 264, Last known committed idx: 263, Time since last communication: 0.000s
I20240501 19:56:33.777629 24755 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 263. Preceding OpId from leader: term: 1 index: 264. (index mismatch)
I20240501 19:56:33.777709 25750 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 264, Last known committed idx: 263, Time since last communication: 0.000s
I20240501 19:56:33.777922 23975 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 263. Preceding OpId from leader: term: 1 index: 264. (index mismatch)
I20240501 19:56:33.778482 26043 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 264, Last known committed idx: 263, Time since last communication: 0.000s
I20240501 19:56:33.778615 26044 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 264, Last known committed idx: 263, Time since last communication: 0.000s
I20240501 19:56:33.780182 25750 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.264: config changed from index 82 to 264, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
W20240501 19:56:33.780822 23260 consensus_peers.cc:485] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd -> Peer d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8:35395): Couldn't send request to peer d6f0e2c51e3d4598b54766125f1d811f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c9623f516f84445ade4bf06eb05673f. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:33.780689 24755 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.264: config changed from index 82 to 264, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.780664 23975 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.264: config changed from index 82 to 264, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.782466 23849 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.264: config changed from index 82 to 264, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.782876 23581 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.264: config changed from index 82 to 264, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.784830 23179 catalog_manager.cc:5484] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index 82 to 264, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:33.792106 23975 consensus_queue.cc:237] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 394, Committed index: 394, Last appended: 1.394, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 395 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:33.793632 24755 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 394. Preceding OpId from leader: term: 1 index: 395. (index mismatch)
I20240501 19:56:33.793874 23326 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 394. Preceding OpId from leader: term: 1 index: 395. (index mismatch)
I20240501 19:56:33.794198 26059 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 395, Last known committed idx: 394, Time since last communication: 0.000s
I20240501 19:56:33.794298 23583 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 394. Preceding OpId from leader: term: 1 index: 395. (index mismatch)
I20240501 19:56:33.794584 23849 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 394. Preceding OpId from leader: term: 1 index: 395. (index mismatch)
I20240501 19:56:33.794612 26061 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 395, Last known committed idx: 394, Time since last communication: 0.000s
I20240501 19:56:33.795480 26061 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 395, Last known committed idx: 394, Time since last communication: 0.000s
I20240501 19:56:33.795743 26061 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 395, Last known committed idx: 394, Time since last communication: 0.000s
I20240501 19:56:33.797494 24755 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.395: config changed from index 187 to 395, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 395 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.797703 23849 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.395: config changed from index 187 to 395, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 395 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.799947 23180 catalog_manager.cc:5484] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index 187 to 395, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "4f51f982ad284c22aae01c4541a621f1" committed_config { opid_index: 395 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.800451 26061 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 1 LEADER]: Committing config change with OpId 1.395: config changed from index 187 to 395, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 395 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.803342 23326 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.395: config changed from index 187 to 395, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 395 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.803432 23583 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.395: config changed from index 187 to 395, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 395 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
W20240501 19:56:33.809619 23915 consensus_peers.cc:485] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 -> Peer d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8:35395): Couldn't send request to peer d6f0e2c51e3d4598b54766125f1d811f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 78b919c4db7745fd87151d8834866e26. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:33.814967 23719 consensus_queue.cc:237] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 340, Committed index: 340, Last appended: 1.340, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:33.816299 23583 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 340. Preceding OpId from leader: term: 1 index: 341. (index mismatch)
I20240501 19:56:33.816407 23453 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 340. Preceding OpId from leader: term: 1 index: 341. (index mismatch)
I20240501 19:56:33.816494 24755 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 340. Preceding OpId from leader: term: 1 index: 341. (index mismatch)
W20240501 19:56:33.816516 23653 consensus_peers.cc:485] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 -> Peer d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8:35395): Couldn't send request to peer d6f0e2c51e3d4598b54766125f1d811f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 292aa245734c41d9814b8347d543b8fb. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:33.816694 26027 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 341, Last known committed idx: 340, Time since last communication: 0.000s
I20240501 19:56:33.816861 23849 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 340. Preceding OpId from leader: term: 1 index: 341. (index mismatch)
I20240501 19:56:33.816833 26073 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 341, Last known committed idx: 340, Time since last communication: 0.000s
I20240501 19:56:33.817148 26027 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 341, Last known committed idx: 340, Time since last communication: 0.000s
I20240501 19:56:33.817433 25696 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 341, Last known committed idx: 340, Time since last communication: 0.000s
I20240501 19:56:33.818689 26027 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.341: config changed from index 153 to 341, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.819403 24755 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.341: config changed from index 153 to 341, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.819556 23849 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.341: config changed from index 153 to 341, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.819648 23583 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.341: config changed from index 153 to 341, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.820883 23453 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.341: config changed from index 153 to 341, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.821837 23182 catalog_manager.cc:5484] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 reported cstate change: config changed from index 153 to 341, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.828873 23849 consensus_queue.cc:237] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:33.830196 23453 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:33.830479 24755 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:33.830479 23719 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:33.830561 25797 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:33.830977 26084 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:33.831077 23975 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:33.831211 25935 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:33.831410 26086 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:33.833161 25797 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.833545 24755 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.835304 23975 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.836227 23179 catalog_manager.cc:5484] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index 4 to 5, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:33.836246 23453 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:33.836616 23719 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
W20240501 19:56:33.838016 23784 consensus_peers.cc:485] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 -> Peer d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8:35395): Couldn't send request to peer d6f0e2c51e3d4598b54766125f1d811f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 44f96d2d427d43008630eb7bcc39caf9. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:34.306002 26162 ts_tablet_manager.cc:902] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f: Initiating tablet copy from peer ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723)
I20240501 19:56:34.306782 26162 tablet_copy_client.cc:308] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.5:44723
I20240501 19:56:34.310206 23870 tablet_copy_service.cc:140] P ee9bc706484e48fb8ff739a61cabe858: Received BeginTabletCopySession request for tablet 44f96d2d427d43008630eb7bcc39caf9 from peer d6f0e2c51e3d4598b54766125f1d811f ({username='slave'} at 127.20.210.8:53705)
I20240501 19:56:34.310326 23870 tablet_copy_service.cc:161] P ee9bc706484e48fb8ff739a61cabe858: Beginning new tablet copy session on tablet 44f96d2d427d43008630eb7bcc39caf9 from peer d6f0e2c51e3d4598b54766125f1d811f at {username='slave'} at 127.20.210.8:53705: session id = d6f0e2c51e3d4598b54766125f1d811f-44f96d2d427d43008630eb7bcc39caf9
I20240501 19:56:34.311143 26173 ts_tablet_manager.cc:902] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f: Initiating tablet copy from peer ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481)
I20240501 19:56:34.311199 23870 tablet_copy_source_session.cc:205] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:34.311635 26173 tablet_copy_client.cc:308] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:35481
I20240501 19:56:34.314432 26162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f96d2d427d43008630eb7bcc39caf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:34.314779 23346 tablet_copy_service.cc:140] P ad9452840e4e4184a640fb63dab754fd: Received BeginTabletCopySession request for tablet 1c9623f516f84445ade4bf06eb05673f from peer d6f0e2c51e3d4598b54766125f1d811f ({username='slave'} at 127.20.210.8:54995)
I20240501 19:56:34.314877 23346 tablet_copy_service.cc:161] P ad9452840e4e4184a640fb63dab754fd: Beginning new tablet copy session on tablet 1c9623f516f84445ade4bf06eb05673f from peer d6f0e2c51e3d4598b54766125f1d811f at {username='slave'} at 127.20.210.8:54995: session id = d6f0e2c51e3d4598b54766125f1d811f-1c9623f516f84445ade4bf06eb05673f
I20240501 19:56:34.315716 23346 tablet_copy_source_session.cc:205] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:34.316694 26173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c9623f516f84445ade4bf06eb05673f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:34.319875 26162 tablet_copy_client.cc:783] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:34.320057 26173 tablet_copy_client.cc:783] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:34.320055 26162 tablet_copy_client.cc:647] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:34.320240 26173 tablet_copy_client.cc:647] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:34.321928 26162 tablet_copy_client.cc:523] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:34.322187 26173 tablet_copy_client.cc:523] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:34.323357 26162 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap starting.
I20240501 19:56:34.323359 26173 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap starting.
I20240501 19:56:34.323933 26178 ts_tablet_manager.cc:902] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f: Initiating tablet copy from peer 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657)
I20240501 19:56:34.324409 26178 tablet_copy_client.cc:308] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:44657
I20240501 19:56:34.328317 23739 tablet_copy_service.cc:140] P 6e47806645ae47278e35600cc67d5700: Received BeginTabletCopySession request for tablet 6f549fed054d433381ac6d38d64c74bb from peer d6f0e2c51e3d4598b54766125f1d811f ({username='slave'} at 127.20.210.8:48507)
I20240501 19:56:34.328421 23739 tablet_copy_service.cc:161] P 6e47806645ae47278e35600cc67d5700: Beginning new tablet copy session on tablet 6f549fed054d433381ac6d38d64c74bb from peer d6f0e2c51e3d4598b54766125f1d811f at {username='slave'} at 127.20.210.8:48507: session id = d6f0e2c51e3d4598b54766125f1d811f-6f549fed054d433381ac6d38d64c74bb
I20240501 19:56:34.328620 26162 log.cc:824] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:34.329350 23739 tablet_copy_source_session.cc:205] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:34.330062 26178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f549fed054d433381ac6d38d64c74bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:34.331964 26162 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:34.332239 26162 tablet_bootstrap.cc:492] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap complete.
I20240501 19:56:34.332252 26178 tablet_copy_client.cc:783] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:34.332532 26162 ts_tablet_manager.cc:1367] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.001s
I20240501 19:56:34.332969 26178 tablet_copy_client.cc:647] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:34.333649 26184 ts_tablet_manager.cc:902] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f: Initiating tablet copy from peer 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667)
I20240501 19:56:34.334162 26184 tablet_copy_client.cc:308] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.6:37667
I20240501 19:56:34.335533 26162 raft_consensus.cc:348] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:34.335772 26162 raft_consensus.cc:729] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d6f0e2c51e3d4598b54766125f1d811f, State: Initialized, Role: LEARNER
I20240501 19:56:34.336118 26162 consensus_queue.cc:260] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:34.336668 25930 heartbeater.cc:503] Master 127.20.210.62:37935 was elected leader, sending a full tablet report...
I20240501 19:56:34.336912 26162 ts_tablet_manager.cc:1398] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20240501 19:56:34.338079 23870 tablet_copy_service.cc:342] P ee9bc706484e48fb8ff739a61cabe858: Request end of tablet copy session d6f0e2c51e3d4598b54766125f1d811f-44f96d2d427d43008630eb7bcc39caf9 received from {username='slave'} at 127.20.210.8:53705
I20240501 19:56:34.338172 23870 tablet_copy_service.cc:433] P ee9bc706484e48fb8ff739a61cabe858: ending tablet copy session d6f0e2c51e3d4598b54766125f1d811f-44f96d2d427d43008630eb7bcc39caf9 on tablet 44f96d2d427d43008630eb7bcc39caf9 with peer d6f0e2c51e3d4598b54766125f1d811f
I20240501 19:56:34.338451 26178 tablet_copy_client.cc:523] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:34.339278 24001 tablet_copy_service.cc:140] P 4f51f982ad284c22aae01c4541a621f1: Received BeginTabletCopySession request for tablet 78b919c4db7745fd87151d8834866e26 from peer d6f0e2c51e3d4598b54766125f1d811f ({username='slave'} at 127.20.210.8:57807)
I20240501 19:56:34.339375 24001 tablet_copy_service.cc:161] P 4f51f982ad284c22aae01c4541a621f1: Beginning new tablet copy session on tablet 78b919c4db7745fd87151d8834866e26 from peer d6f0e2c51e3d4598b54766125f1d811f at {username='slave'} at 127.20.210.8:57807: session id = d6f0e2c51e3d4598b54766125f1d811f-78b919c4db7745fd87151d8834866e26
I20240501 19:56:34.339742 26178 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap starting.
I20240501 19:56:34.340368 24001 tablet_copy_source_session.cc:205] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:34.341507 26184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78b919c4db7745fd87151d8834866e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:34.343319 26184 tablet_copy_client.cc:783] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:34.343449 26184 tablet_copy_client.cc:647] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:34.348105 26184 tablet_copy_client.cc:523] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:34.349123 26184 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap starting.
I20240501 19:56:34.434079 26162 ts_tablet_manager.cc:902] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f: Initiating tablet copy from peer 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657)
I20240501 19:56:34.434805 26162 tablet_copy_client.cc:308] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:44657
I20240501 19:56:34.435349 23739 tablet_copy_service.cc:140] P 6e47806645ae47278e35600cc67d5700: Received BeginTabletCopySession request for tablet 292aa245734c41d9814b8347d543b8fb from peer d6f0e2c51e3d4598b54766125f1d811f ({username='slave'} at 127.20.210.8:48507)
I20240501 19:56:34.435444 23739 tablet_copy_service.cc:161] P 6e47806645ae47278e35600cc67d5700: Beginning new tablet copy session on tablet 292aa245734c41d9814b8347d543b8fb from peer d6f0e2c51e3d4598b54766125f1d811f at {username='slave'} at 127.20.210.8:48507: session id = d6f0e2c51e3d4598b54766125f1d811f-292aa245734c41d9814b8347d543b8fb
I20240501 19:56:34.436419 23739 tablet_copy_source_session.cc:205] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:34.438349 26162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292aa245734c41d9814b8347d543b8fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:34.440294 26162 tablet_copy_client.cc:783] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:34.440428 26162 tablet_copy_client.cc:647] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:34.443527 26162 tablet_copy_client.cc:523] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:34.444618 26162 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap starting.
I20240501 19:56:34.549652 26173 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap replayed 1/1 log segments. Stats: ops{read=264 overwritten=0 applied=264 ignored=0} inserts{seen=933 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:34.549937 26173 tablet_bootstrap.cc:492] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap complete.
I20240501 19:56:34.550138 26173 ts_tablet_manager.cc:1367] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f: Time spent bootstrapping tablet: real 0.227s user 0.185s sys 0.007s
I20240501 19:56:34.550714 26173 raft_consensus.cc:348] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:34.550841 26173 raft_consensus.cc:729] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d6f0e2c51e3d4598b54766125f1d811f, State: Initialized, Role: LEARNER
I20240501 19:56:34.550940 26173 consensus_queue.cc:260] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 264, Last appended: 1.264, Last appended by leader: 264, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:34.551283 26173 ts_tablet_manager.cc:1398] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 19:56:34.552033 26178 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap replayed 1/1 log segments. Stats: ops{read=261 overwritten=0 applied=261 ignored=0} inserts{seen=930 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:34.552243 26178 tablet_bootstrap.cc:492] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap complete.
I20240501 19:56:34.552400 26178 ts_tablet_manager.cc:1367] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f: Time spent bootstrapping tablet: real 0.213s user 0.160s sys 0.004s
I20240501 19:56:34.552771 26178 raft_consensus.cc:348] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:34.552872 26178 raft_consensus.cc:729] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d6f0e2c51e3d4598b54766125f1d811f, State: Initialized, Role: LEARNER
I20240501 19:56:34.552949 26178 consensus_queue.cc:260] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 261, Last appended: 1.261, Last appended by leader: 261, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:34.553216 26178 ts_tablet_manager.cc:1398] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 19:56:34.553603 23739 tablet_copy_service.cc:342] P 6e47806645ae47278e35600cc67d5700: Request end of tablet copy session d6f0e2c51e3d4598b54766125f1d811f-6f549fed054d433381ac6d38d64c74bb received from {username='slave'} at 127.20.210.8:48507
I20240501 19:56:34.553675 23739 tablet_copy_service.cc:433] P 6e47806645ae47278e35600cc67d5700: ending tablet copy session d6f0e2c51e3d4598b54766125f1d811f-6f549fed054d433381ac6d38d64c74bb on tablet 6f549fed054d433381ac6d38d64c74bb with peer d6f0e2c51e3d4598b54766125f1d811f
I20240501 19:56:34.556699 23346 tablet_copy_service.cc:342] P ad9452840e4e4184a640fb63dab754fd: Request end of tablet copy session d6f0e2c51e3d4598b54766125f1d811f-1c9623f516f84445ade4bf06eb05673f received from {username='slave'} at 127.20.210.8:54995
I20240501 19:56:34.556769 23346 tablet_copy_service.cc:433] P ad9452840e4e4184a640fb63dab754fd: ending tablet copy session d6f0e2c51e3d4598b54766125f1d811f-1c9623f516f84445ade4bf06eb05673f on tablet 1c9623f516f84445ade4bf06eb05673f with peer d6f0e2c51e3d4598b54766125f1d811f
I20240501 19:56:34.625357 26184 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap replayed 1/1 log segments. Stats: ops{read=395 overwritten=0 applied=395 ignored=0} inserts{seen=1064 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:34.625607 26184 tablet_bootstrap.cc:492] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap complete.
I20240501 19:56:34.625788 26184 ts_tablet_manager.cc:1367] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f: Time spent bootstrapping tablet: real 0.277s user 0.230s sys 0.027s
I20240501 19:56:34.626152 26184 raft_consensus.cc:348] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 395 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:34.626278 26184 raft_consensus.cc:729] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d6f0e2c51e3d4598b54766125f1d811f, State: Initialized, Role: LEARNER
I20240501 19:56:34.626377 26184 consensus_queue.cc:260] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 395, Last appended: 1.395, Last appended by leader: 395, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 395 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:34.626632 26184 ts_tablet_manager.cc:1398] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f: Time spent starting tablet: real 0.001s user 0.000s sys 0.004s
I20240501 19:56:34.627213 24001 tablet_copy_service.cc:342] P 4f51f982ad284c22aae01c4541a621f1: Request end of tablet copy session d6f0e2c51e3d4598b54766125f1d811f-78b919c4db7745fd87151d8834866e26 received from {username='slave'} at 127.20.210.8:57807
I20240501 19:56:34.627300 24001 tablet_copy_service.cc:433] P 4f51f982ad284c22aae01c4541a621f1: ending tablet copy session d6f0e2c51e3d4598b54766125f1d811f-78b919c4db7745fd87151d8834866e26 on tablet 78b919c4db7745fd87151d8834866e26 with peer d6f0e2c51e3d4598b54766125f1d811f
I20240501 19:56:34.675779 26162 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap replayed 1/1 log segments. Stats: ops{read=341 overwritten=0 applied=341 ignored=0} inserts{seen=969 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:34.676048 26162 tablet_bootstrap.cc:492] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap complete.
I20240501 19:56:34.676210 26162 ts_tablet_manager.cc:1367] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f: Time spent bootstrapping tablet: real 0.232s user 0.179s sys 0.021s
I20240501 19:56:34.676523 26162 raft_consensus.cc:348] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:34.676628 26162 raft_consensus.cc:729] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d6f0e2c51e3d4598b54766125f1d811f, State: Initialized, Role: LEARNER
I20240501 19:56:34.676723 26162 consensus_queue.cc:260] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 341, Last appended: 1.341, Last appended by leader: 341, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 341 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:34.676946 26162 ts_tablet_manager.cc:1398] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:34.677359 23739 tablet_copy_service.cc:342] P 6e47806645ae47278e35600cc67d5700: Request end of tablet copy session d6f0e2c51e3d4598b54766125f1d811f-292aa245734c41d9814b8347d543b8fb received from {username='slave'} at 127.20.210.8:48507
I20240501 19:56:34.677430 23739 tablet_copy_service.cc:433] P 6e47806645ae47278e35600cc67d5700: ending tablet copy session d6f0e2c51e3d4598b54766125f1d811f-292aa245734c41d9814b8347d543b8fb on tablet 292aa245734c41d9814b8347d543b8fb with peer d6f0e2c51e3d4598b54766125f1d811f
I20240501 19:56:34.710587 25884 raft_consensus.cc:1203] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.263->[1.264-1.264] Dedup: 1.264->[]
I20240501 19:56:34.811858 25884 raft_consensus.cc:1203] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:56:34.838264 25884 raft_consensus.cc:1203] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.394->[1.395-1.395] Dedup: 1.395->[]
I20240501 19:56:34.866539 25884 raft_consensus.cc:1203] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.260->[1.261-1.261] Dedup: 1.261->[]
I20240501 19:56:35.026798 25884 raft_consensus.cc:1203] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.340->[1.341-1.341] Dedup: 1.341->[]
I20240501 19:56:35.255101 25935 raft_consensus.cc:1050] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858: attempting to promote NON_VOTER d6f0e2c51e3d4598b54766125f1d811f to VOTER
I20240501 19:56:35.255545 25935 consensus_queue.cc:237] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:35.256976 23453 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:35.257031 23975 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:35.257050 24755 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:35.257048 23719 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:35.257443 26308 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:35.257476 25884 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:35.257581 26309 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:35.257753 26307 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:35.257961 25935 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:35.258601 26307 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:35.260314 26307 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.260702 23719 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.261083 23975 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.262719 23453 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.263044 24755 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.263571 23184 catalog_manager.cc:5484] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index 5 to 6, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:35.265199 25884 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.270027 23849 consensus_queue.cc:237] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:35.271135 24755 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:35.271150 25884 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:35.271150 23975 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:35.271140 23453 raft_consensus.cc:1261] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ee9bc706484e48fb8ff739a61cabe858: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:35.271548 25935 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:35.271679 26307 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:35.271803 26308 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:35.272243 26308 consensus_queue.cc:1035] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:35.273802 26307 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P ee9bc706484e48fb8ff739a61cabe858 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.274888 24755 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.275585 23168 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 44f96d2d427d43008630eb7bcc39caf9 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:35.274933 23975 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.275838 23453 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.277212 23180 catalog_manager.cc:5484] T 44f96d2d427d43008630eb7bcc39caf9 P 3b3f502cbeb1451087a06f95a3d93f2b reported cstate change: config changed from index 6 to 7, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "ee9bc706484e48fb8ff739a61cabe858" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.277901 25884 raft_consensus.cc:2932] T 44f96d2d427d43008630eb7bcc39caf9 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 6e47806645ae47278e35600cc67d5700 (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.279554 23698 tablet_service.cc:1514] Processing DeleteTablet for tablet 44f96d2d427d43008630eb7bcc39caf9 with delete_type TABLET_DATA_TOMBSTONED (TS 6e47806645ae47278e35600cc67d5700 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:54842
I20240501 19:56:35.279812 26320 tablet_replica.cc:330] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700: stopping tablet replica
I20240501 19:56:35.279965 26320 raft_consensus.cc:2229] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:35.280128 26320 raft_consensus.cc:2258] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:35.281327 26320 ts_tablet_manager.cc:1875] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:35.283008 26320 ts_tablet_manager.cc:1888] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:56:35.283066 26320 log.cc:1196] T 44f96d2d427d43008630eb7bcc39caf9 P 6e47806645ae47278e35600cc67d5700: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/44f96d2d427d43008630eb7bcc39caf9
I20240501 19:56:35.283459 23171 catalog_manager.cc:4830] TS 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657): tablet 44f96d2d427d43008630eb7bcc39caf9 (table TestTable [id=e130370886f346e289c105d0836dc0da]) successfully deleted
I20240501 19:56:35.319589 26193 raft_consensus.cc:1050] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700: attempting to promote NON_VOTER d6f0e2c51e3d4598b54766125f1d811f to VOTER
I20240501 19:56:35.319926 26193 consensus_queue.cc:237] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 261, Committed index: 261, Last appended: 1.261, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 262 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:35.321332 23326 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 261. Preceding OpId from leader: term: 1 index: 262. (index mismatch)
I20240501 19:56:35.321352 25884 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 261. Preceding OpId from leader: term: 1 index: 262. (index mismatch)
I20240501 19:56:35.321453 23583 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 261. Preceding OpId from leader: term: 1 index: 262. (index mismatch)
I20240501 19:56:35.321534 23849 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 261. Preceding OpId from leader: term: 1 index: 262. (index mismatch)
I20240501 19:56:35.321779 26322 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 262, Last known committed idx: 261, Time since last communication: 0.000s
I20240501 19:56:35.321944 25696 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 262, Last known committed idx: 261, Time since last communication: 0.000s
I20240501 19:56:35.322058 23453 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 261. Preceding OpId from leader: term: 1 index: 262. (index mismatch)
I20240501 19:56:35.322129 26193 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 262, Last known committed idx: 261, Time since last communication: 0.000s
I20240501 19:56:35.322422 26255 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 262, Last known committed idx: 261, Time since last communication: 0.000s
I20240501 19:56:35.322567 26323 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 262, Last known committed idx: 261, Time since last communication: 0.000s
I20240501 19:56:35.324529 26045 raft_consensus.cc:1050] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd: attempting to promote NON_VOTER d6f0e2c51e3d4598b54766125f1d811f to VOTER
I20240501 19:56:35.325037 26045 consensus_queue.cc:237] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 264, Committed index: 264, Last appended: 1.264, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:35.325063 26323 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.262: config changed from index 261 to 262, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 262 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.325531 23583 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.262: config changed from index 261 to 262, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 262 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.325706 25884 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.262: config changed from index 261 to 262, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 262 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.327319 23849 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.262: config changed from index 261 to 262, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 262 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.328217 23178 catalog_manager.cc:5484] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f reported cstate change: config changed from index 261 to 262, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 262 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.328635 23845 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 264. Preceding OpId from leader: term: 1 index: 265. (index mismatch)
I20240501 19:56:35.328913 23326 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.262: config changed from index 261 to 262, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 262 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.329655 23453 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.262: config changed from index 261 to 262, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 262 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.331097 24755 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 264. Preceding OpId from leader: term: 1 index: 265. (index mismatch)
I20240501 19:56:35.331126 23975 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 264. Preceding OpId from leader: term: 1 index: 265. (index mismatch)
I20240501 19:56:35.331529 23583 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 264. Preceding OpId from leader: term: 1 index: 265. (index mismatch)
I20240501 19:56:35.331586 25884 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 264. Preceding OpId from leader: term: 1 index: 265. (index mismatch)
I20240501 19:56:35.331914 26332 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 265, Last known committed idx: 264, Time since last communication: 0.000s
I20240501 19:56:35.332376 26333 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 265, Last known committed idx: 264, Time since last communication: 0.000s
I20240501 19:56:35.332598 26045 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 265, Last known committed idx: 264, Time since last communication: 0.000s
I20240501 19:56:35.332763 26334 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 265, Last known committed idx: 264, Time since last communication: 0.000s
I20240501 19:56:35.332964 26329 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 265, Last known committed idx: 264, Time since last communication: 0.000s
I20240501 19:56:35.333993 23719 consensus_queue.cc:237] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 262, Committed index: 262, Last appended: 1.262, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:35.335515 25884 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 262. Preceding OpId from leader: term: 1 index: 263. (index mismatch)
I20240501 19:56:35.335515 23583 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 262. Preceding OpId from leader: term: 1 index: 263. (index mismatch)
I20240501 19:56:35.335860 25696 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 263, Last known committed idx: 262, Time since last communication: 0.000s
I20240501 19:56:35.336027 26193 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 263, Last known committed idx: 262, Time since last communication: 0.000s
I20240501 19:56:35.337033 23326 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 262. Preceding OpId from leader: term: 1 index: 263. (index mismatch)
I20240501 19:56:35.337569 25696 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 263, Last known committed idx: 262, Time since last communication: 0.000s
I20240501 19:56:35.338275 23845 raft_consensus.cc:1261] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 262. Preceding OpId from leader: term: 1 index: 263. (index mismatch)
I20240501 19:56:35.339488 26193 consensus_queue.cc:1035] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 263, Last known committed idx: 262, Time since last communication: 0.000s
I20240501 19:56:35.339727 26045 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.265: config changed from index 264 to 265, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.339829 26193 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.263: config changed from index 262 to 263, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.341054 23326 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.263: config changed from index 262 to 263, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.342213 23975 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.265: config changed from index 264 to 265, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.342752 23177 catalog_manager.cc:5484] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index 264 to 265, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:35.342844 23581 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.263: config changed from index 262 to 263, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.343540 24755 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.265: config changed from index 264 to 265, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.344821 23845 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.263: config changed from index 262 to 263, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.345187 23583 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.265: config changed from index 264 to 265, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.345368 23171 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6f549fed054d433381ac6d38d64c74bb with cas_config_opid_index 262: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:35.346658 23184 catalog_manager.cc:5484] T 6f549fed054d433381ac6d38d64c74bb P e8ddd74cfd954c7a8c2d284c335d7826 reported cstate change: config changed from index 262 to 263, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.347661 25883 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.265: config changed from index 264 to 265, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.347702 25884 raft_consensus.cc:2932] T 6f549fed054d433381ac6d38d64c74bb P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.263: config changed from index 262 to 263, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.350134 23326 consensus_queue.cc:237] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 265, Committed index: 265, Last appended: 1.265, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 266 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:35.350891 23435 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f549fed054d433381ac6d38d64c74bb with delete_type TABLET_DATA_TOMBSTONED (TS 0496e037179e4b3ca72aead781c25d5a not found in new config with opid_index 263) from {username='slave'} at 127.0.0.1:49870
I20240501 19:56:35.351562 25883 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 265. Preceding OpId from leader: term: 1 index: 266. (index mismatch)
I20240501 19:56:35.351563 23849 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 265. Preceding OpId from leader: term: 1 index: 266. (index mismatch)
I20240501 19:56:35.351651 23975 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 265. Preceding OpId from leader: term: 1 index: 266. (index mismatch)
I20240501 19:56:35.352046 24755 raft_consensus.cc:1261] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 265. Preceding OpId from leader: term: 1 index: 266. (index mismatch)
I20240501 19:56:35.352074 26045 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 266, Last known committed idx: 265, Time since last communication: 0.000s
I20240501 19:56:35.352252 26341 tablet_replica.cc:330] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a: stopping tablet replica
I20240501 19:56:35.352270 26334 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 266, Last known committed idx: 265, Time since last communication: 0.000s
I20240501 19:56:35.352388 26341 raft_consensus.cc:2229] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:35.352396 26333 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 266, Last known committed idx: 264, Time since last communication: 0.000s
I20240501 19:56:35.352595 26341 raft_consensus.cc:2258] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:35.352888 26333 consensus_queue.cc:1035] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 266, Last known committed idx: 265, Time since last communication: 0.000s
I20240501 19:56:35.353173 23849 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.265: config changed from index 264 to 265, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.356451 26329 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.266: config changed from index 265 to 266, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 266 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.356778 23845 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.266: config changed from index 265 to 266, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 266 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.357493 24755 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.266: config changed from index 265 to 266, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 266 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.358184 23170 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1c9623f516f84445ade4bf06eb05673f with cas_config_opid_index 265: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:35.358603 25883 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.266: config changed from index 265 to 266, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 266 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.358875 23975 raft_consensus.cc:2932] T 1c9623f516f84445ade4bf06eb05673f P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.266: config changed from index 265 to 266, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 266 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.359066 23180 catalog_manager.cc:5484] T 1c9623f516f84445ade4bf06eb05673f P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index 265 to 266, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 266 OBSOLETE_local: false peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:35.359881 26341 ts_tablet_manager.cc:1875] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:35.361157 23568 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c9623f516f84445ade4bf06eb05673f with delete_type TABLET_DATA_TOMBSTONED (TS e8ddd74cfd954c7a8c2d284c335d7826 not found in new config with opid_index 266) from {username='slave'} at 127.0.0.1:46492
I20240501 19:56:35.362226 26341 ts_tablet_manager.cc:1888] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.262
I20240501 19:56:35.362370 26341 log.cc:1196] T 6f549fed054d433381ac6d38d64c74bb P 0496e037179e4b3ca72aead781c25d5a: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/6f549fed054d433381ac6d38d64c74bb
I20240501 19:56:35.362876 23168 catalog_manager.cc:4830] TS 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367): tablet 6f549fed054d433381ac6d38d64c74bb (table rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]) successfully deleted
I20240501 19:56:35.363785 26342 tablet_replica.cc:330] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826: stopping tablet replica
I20240501 19:56:35.363948 26342 raft_consensus.cc:2229] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:35.364210 26342 raft_consensus.cc:2258] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:35.365523 26342 ts_tablet_manager.cc:1875] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:35.367110 26342 ts_tablet_manager.cc:1888] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.265
I20240501 19:56:35.367177 26342 log.cc:1196] T 1c9623f516f84445ade4bf06eb05673f P e8ddd74cfd954c7a8c2d284c335d7826: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/1c9623f516f84445ade4bf06eb05673f
I20240501 19:56:35.367609 23171 catalog_manager.cc:4830] TS e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279): tablet 1c9623f516f84445ade4bf06eb05673f (table rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]) successfully deleted
I20240501 19:56:35.418833 26058 raft_consensus.cc:1050] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1: attempting to promote NON_VOTER d6f0e2c51e3d4598b54766125f1d811f to VOTER
I20240501 19:56:35.419191 26058 consensus_queue.cc:237] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 395, Committed index: 395, Last appended: 1.395, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 396 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:35.420430 24755 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 395. Preceding OpId from leader: term: 1 index: 396. (index mismatch)
I20240501 19:56:35.420696 25883 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 395. Preceding OpId from leader: term: 1 index: 396. (index mismatch)
I20240501 19:56:35.420698 23583 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 395. Preceding OpId from leader: term: 1 index: 396. (index mismatch)
I20240501 19:56:35.420884 23845 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 395. Preceding OpId from leader: term: 1 index: 396. (index mismatch)
I20240501 19:56:35.420922 23326 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 395. Preceding OpId from leader: term: 1 index: 396. (index mismatch)
I20240501 19:56:35.421088 26347 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 396, Last known committed idx: 395, Time since last communication: 0.000s
I20240501 19:56:35.421320 26343 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 396, Last known committed idx: 395, Time since last communication: 0.000s
I20240501 19:56:35.421450 26058 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 396, Last known committed idx: 395, Time since last communication: 0.000s
I20240501 19:56:35.421629 26345 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 396, Last known committed idx: 395, Time since last communication: 0.000s
I20240501 19:56:35.421756 26346 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 396, Last known committed idx: 395, Time since last communication: 0.000s
I20240501 19:56:35.423954 26347 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 1 LEADER]: Committing config change with OpId 1.396: config changed from index 395 to 396, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 396 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.424621 23326 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.396: config changed from index 395 to 396, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 396 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.424718 25883 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.396: config changed from index 395 to 396, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 396 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.426204 23583 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.396: config changed from index 395 to 396, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 396 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.427150 23180 catalog_manager.cc:5484] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f reported cstate change: config changed from index 395 to 396, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4f51f982ad284c22aae01c4541a621f1" committed_config { opid_index: 396 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.427441 24755 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.396: config changed from index 395 to 396, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 396 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.428790 23845 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.396: config changed from index 395 to 396, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 396 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } attrs { replace: true } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.431634 23975 consensus_queue.cc:237] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 396, Committed index: 396, Last appended: 1.396, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 397 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:35.433005 23583 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 396. Preceding OpId from leader: term: 1 index: 397. (index mismatch)
I20240501 19:56:35.433316 23845 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 396. Preceding OpId from leader: term: 1 index: 397. (index mismatch)
I20240501 19:56:35.433315 26346 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 397, Last known committed idx: 396, Time since last communication: 0.000s
I20240501 19:56:35.433611 25883 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 396. Preceding OpId from leader: term: 1 index: 397. (index mismatch)
I20240501 19:56:35.433696 24755 raft_consensus.cc:1261] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer 4f51f982ad284c22aae01c4541a621f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 396. Preceding OpId from leader: term: 1 index: 397. (index mismatch)
I20240501 19:56:35.433718 26346 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 397, Last known committed idx: 396, Time since last communication: 0.000s
I20240501 19:56:35.433955 26347 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 397, Last known committed idx: 396, Time since last communication: 0.000s
I20240501 19:56:35.434284 26346 consensus_queue.cc:1035] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 397, Last known committed idx: 396, Time since last communication: 0.000s
I20240501 19:56:35.436182 26343 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 [term 1 LEADER]: Committing config change with OpId 1.397: config changed from index 396 to 397, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 397 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.437608 23169 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 78b919c4db7745fd87151d8834866e26 with cas_config_opid_index 396: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:35.437927 23583 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.397: config changed from index 396 to 397, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 397 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.438323 23845 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.397: config changed from index 396 to 397, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 397 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.438783 24755 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.397: config changed from index 396 to 397, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 397 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.439404 25883 raft_consensus.cc:2932] T 78b919c4db7745fd87151d8834866e26 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.397: config changed from index 396 to 397, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New config: { opid_index: 397 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.439931 23179 catalog_manager.cc:5484] T 78b919c4db7745fd87151d8834866e26 P 4f51f982ad284c22aae01c4541a621f1 reported cstate change: config changed from index 396 to 397, VOTER ad9452840e4e4184a640fb63dab754fd (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "4f51f982ad284c22aae01c4541a621f1" committed_config { opid_index: 397 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:35.443363 23305 tablet_service.cc:1514] Processing DeleteTablet for tablet 78b919c4db7745fd87151d8834866e26 with delete_type TABLET_DATA_TOMBSTONED (TS ad9452840e4e4184a640fb63dab754fd not found in new config with opid_index 397) from {username='slave'} at 127.0.0.1:55042
I20240501 19:56:35.443711 26358 tablet_replica.cc:330] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd: stopping tablet replica
I20240501 19:56:35.443881 26358 raft_consensus.cc:2229] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:35.444061 26358 raft_consensus.cc:2258] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:35.445590 26358 ts_tablet_manager.cc:1875] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:35.447432 26358 ts_tablet_manager.cc:1888] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.396
I20240501 19:56:35.447497 26358 log.cc:1196] T 78b919c4db7745fd87151d8834866e26 P ad9452840e4e4184a640fb63dab754fd: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/78b919c4db7745fd87151d8834866e26
I20240501 19:56:35.447966 23170 catalog_manager.cc:4830] TS ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481): tablet 78b919c4db7745fd87151d8834866e26 (table rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]) successfully deleted
I20240501 19:56:35.472987 23453 consensus_queue.cc:237] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 364, Committed index: 364, Last appended: 1.364, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 365 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
W20240501 19:56:35.477982 23391 consensus_peers.cc:485] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a -> Peer d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8:35395): Couldn't send request to peer d6f0e2c51e3d4598b54766125f1d811f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 87ebb3dfc00d4744960bde9a07c59e23. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:35.478433 23975 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 364. Preceding OpId from leader: term: 1 index: 365. (index mismatch)
I20240501 19:56:35.478755 26387 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 365, Last known committed idx: 364, Time since last communication: 0.000s
I20240501 19:56:35.478865 23326 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 364. Preceding OpId from leader: term: 1 index: 365. (index mismatch)
I20240501 19:56:35.479279 23719 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 364. Preceding OpId from leader: term: 1 index: 365. (index mismatch)
I20240501 19:56:35.479261 26387 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 365, Last known committed idx: 364, Time since last communication: 0.000s
I20240501 19:56:35.479617 23583 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 364. Preceding OpId from leader: term: 1 index: 365. (index mismatch)
I20240501 19:56:35.479887 26387 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 365, Last known committed idx: 364, Time since last communication: 0.000s
I20240501 19:56:35.480546 26388 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 365, Last known committed idx: 364, Time since last communication: 0.000s
I20240501 19:56:35.481530 26387 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Committing config change with OpId 1.365: config changed from index -1 to 365, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 365 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.482621 23719 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.365: config changed from index -1 to 365, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 365 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.483114 23583 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.365: config changed from index -1 to 365, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 365 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.484429 23177 catalog_manager.cc:5484] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a reported cstate change: config changed from index -1 to 365, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: 365 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:35.486624 23975 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.365: config changed from index -1 to 365, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 365 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.491324 23326 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.365: config changed from index -1 to 365, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 365 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.491662 23453 consensus_queue.cc:237] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 258, Committed index: 258, Last appended: 1.258, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 259 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:35.493489 24755 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 258. Preceding OpId from leader: term: 1 index: 259. (index mismatch)
I20240501 19:56:35.493803 26388 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 259, Last known committed idx: 258, Time since last communication: 0.000s
I20240501 19:56:35.494206 23583 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 258. Preceding OpId from leader: term: 1 index: 259. (index mismatch)
I20240501 19:56:35.494334 23975 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 258. Preceding OpId from leader: term: 1 index: 259. (index mismatch)
I20240501 19:56:35.494460 23719 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 258. Preceding OpId from leader: term: 1 index: 259. (index mismatch)
I20240501 19:56:35.495549 26388 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 259, Last known committed idx: 258, Time since last communication: 0.000s
I20240501 19:56:35.495755 25751 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 259, Last known committed idx: 258, Time since last communication: 0.000s
I20240501 19:56:35.495873 26388 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 259, Last known committed idx: 258, Time since last communication: 0.000s
W20240501 19:56:35.496367 23391 consensus_peers.cc:485] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a -> Peer d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8:35395): Couldn't send request to peer d6f0e2c51e3d4598b54766125f1d811f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f701e0b045d540698ba3879954d8b171. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:35.497617 26386 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Committing config change with OpId 1.259: config changed from index 61 to 259, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 259 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.497983 23975 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.259: config changed from index 61 to 259, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 259 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.498014 23719 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.259: config changed from index 61 to 259, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 259 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.498409 23583 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.259: config changed from index 61 to 259, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 259 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.500757 23180 catalog_manager.cc:5484] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 reported cstate change: config changed from index 61 to 259, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: 259 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.501911 24755 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.259: config changed from index 61 to 259, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 259 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.507577 23326 consensus_queue.cc:237] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 253, Committed index: 253, Last appended: 1.253, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:35.508960 24755 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 253. Preceding OpId from leader: term: 1 index: 254. (index mismatch)
I20240501 19:56:35.509140 23975 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 253. Preceding OpId from leader: term: 1 index: 254. (index mismatch)
I20240501 19:56:35.509188 23453 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 253. Preceding OpId from leader: term: 1 index: 254. (index mismatch)
I20240501 19:56:35.509562 26334 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 254, Last known committed idx: 253, Time since last communication: 0.000s
I20240501 19:56:35.509636 23719 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 253. Preceding OpId from leader: term: 1 index: 254. (index mismatch)
I20240501 19:56:35.509707 26329 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 254, Last known committed idx: 253, Time since last communication: 0.000s
I20240501 19:56:35.509879 26045 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 254, Last known committed idx: 253, Time since last communication: 0.000s
I20240501 19:56:35.510001 26331 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 254, Last known committed idx: 253, Time since last communication: 0.000s
W20240501 19:56:35.509876 23260 consensus_peers.cc:485] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd -> Peer d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8:35395): Couldn't send request to peer d6f0e2c51e3d4598b54766125f1d811f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b1a0cebcdad24e34abc11eef9d4d891d. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:35.511602 26045 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.254: config changed from index 78 to 254, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.512007 23453 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.254: config changed from index 78 to 254, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.512082 23719 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.254: config changed from index 78 to 254, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.512197 23975 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.254: config changed from index 78 to 254, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.514214 23179 catalog_manager.cc:5484] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 reported cstate change: config changed from index 78 to 254, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.515125 24755 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.254: config changed from index 78 to 254, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.521586 23326 consensus_queue.cc:237] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 213, Committed index: 213, Last appended: 1.213, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 214 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:35.523064 24755 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 213. Preceding OpId from leader: term: 1 index: 214. (index mismatch)
I20240501 19:56:35.523401 23845 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 213. Preceding OpId from leader: term: 1 index: 214. (index mismatch)
I20240501 19:56:35.523401 26329 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 214, Last known committed idx: 213, Time since last communication: 0.000s
I20240501 19:56:35.523484 23719 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 213. Preceding OpId from leader: term: 1 index: 214. (index mismatch)
I20240501 19:56:35.523722 23583 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 213. Preceding OpId from leader: term: 1 index: 214. (index mismatch)
I20240501 19:56:35.523810 26329 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 214, Last known committed idx: 213, Time since last communication: 0.000s
W20240501 19:56:35.524063 23260 consensus_peers.cc:485] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd -> Peer d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8:35395): Couldn't send request to peer d6f0e2c51e3d4598b54766125f1d811f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a4f98ae45e914fa3981237af149b0666. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:35.524230 26329 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 214, Last known committed idx: 213, Time since last communication: 0.000s
I20240501 19:56:35.524981 26329 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 214, Last known committed idx: 213, Time since last communication: 0.000s
I20240501 19:56:35.526316 26334 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.214: config changed from index 120 to 214, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 214 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.527920 23719 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.214: config changed from index 120 to 214, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 214 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.527870 23845 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.214: config changed from index 120 to 214, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 214 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.528487 23180 catalog_manager.cc:5484] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index 120 to 214, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 214 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:35.528530 23583 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.214: config changed from index 120 to 214, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 214 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.531384 24755 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.214: config changed from index 120 to 214, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 214 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.561656 26255 raft_consensus.cc:1050] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700: attempting to promote NON_VOTER d6f0e2c51e3d4598b54766125f1d811f to VOTER
I20240501 19:56:35.562014 26255 consensus_queue.cc:237] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 341, Committed index: 341, Last appended: 1.341, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 342 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:35.563344 25883 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 341. Preceding OpId from leader: term: 1 index: 342. (index mismatch)
I20240501 19:56:35.563426 23457 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 341. Preceding OpId from leader: term: 1 index: 342. (index mismatch)
I20240501 19:56:35.563419 24755 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 341. Preceding OpId from leader: term: 1 index: 342. (index mismatch)
I20240501 19:56:35.563817 26323 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 342, Last known committed idx: 341, Time since last communication: 0.000s
I20240501 19:56:35.563952 23583 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 341. Preceding OpId from leader: term: 1 index: 342. (index mismatch)
I20240501 19:56:35.563978 23845 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 341. Preceding OpId from leader: term: 1 index: 342. (index mismatch)
I20240501 19:56:35.564049 26322 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 342, Last known committed idx: 341, Time since last communication: 0.000s
I20240501 19:56:35.564352 26323 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 342, Last known committed idx: 341, Time since last communication: 0.000s
I20240501 19:56:35.564484 26255 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 342, Last known committed idx: 341, Time since last communication: 0.000s
I20240501 19:56:35.564746 26323 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 342, Last known committed idx: 341, Time since last communication: 0.000s
I20240501 19:56:35.566572 26255 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.342: config changed from index 341 to 342, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 342 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.566948 23845 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.342: config changed from index 341 to 342, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 342 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.567034 24755 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.342: config changed from index 341 to 342, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 342 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.568686 23583 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.342: config changed from index 341 to 342, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 342 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.568794 25883 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.342: config changed from index 341 to 342, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 342 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.569186 23457 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.342: config changed from index 341 to 342, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 342 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.570395 23184 catalog_manager.cc:5484] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 reported cstate change: config changed from index 341 to 342, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 342 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:35.573012 23719 consensus_queue.cc:237] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 342, Committed index: 342, Last appended: 1.342, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:35.574173 24755 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 342. Preceding OpId from leader: term: 1 index: 343. (index mismatch)
I20240501 19:56:35.574401 25883 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 342. Preceding OpId from leader: term: 1 index: 343. (index mismatch)
I20240501 19:56:35.574800 23457 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 342. Preceding OpId from leader: term: 1 index: 343. (index mismatch)
I20240501 19:56:35.574792 26323 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 343, Last known committed idx: 342, Time since last communication: 0.000s
I20240501 19:56:35.574896 23583 raft_consensus.cc:1261] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 342. Preceding OpId from leader: term: 1 index: 343. (index mismatch)
I20240501 19:56:35.575238 26255 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 343, Last known committed idx: 342, Time since last communication: 0.000s
I20240501 19:56:35.575395 26323 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 343, Last known committed idx: 342, Time since last communication: 0.000s
I20240501 19:56:35.575532 26193 consensus_queue.cc:1035] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 343, Last known committed idx: 342, Time since last communication: 0.000s
I20240501 19:56:35.576825 26323 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.343: config changed from index 342 to 343, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.577270 23457 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.343: config changed from index 342 to 343, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.577318 23583 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.343: config changed from index 342 to 343, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.578197 23171 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 292aa245734c41d9814b8347d543b8fb with cas_config_opid_index 342: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:35.578907 24755 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.343: config changed from index 342 to 343, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.579255 23182 catalog_manager.cc:5484] T 292aa245734c41d9814b8347d543b8fb P 0496e037179e4b3ca72aead781c25d5a reported cstate change: config changed from index 342 to 343, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.579473 25883 raft_consensus.cc:2932] T 292aa245734c41d9814b8347d543b8fb P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.343: config changed from index 342 to 343, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 343 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:35.582253 23830 tablet_service.cc:1514] Processing DeleteTablet for tablet 292aa245734c41d9814b8347d543b8fb with delete_type TABLET_DATA_TOMBSTONED (TS ee9bc706484e48fb8ff739a61cabe858 not found in new config with opid_index 343) from {username='slave'} at 127.0.0.1:57012
I20240501 19:56:35.582605 26457 tablet_replica.cc:330] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858: stopping tablet replica
I20240501 19:56:35.582796 26457 raft_consensus.cc:2229] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:35.583070 26457 raft_consensus.cc:2258] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:35.585060 26457 ts_tablet_manager.cc:1875] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:35.586740 26457 ts_tablet_manager.cc:1888] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.342
I20240501 19:56:35.586802 26457 log.cc:1196] T 292aa245734c41d9814b8347d543b8fb P ee9bc706484e48fb8ff739a61cabe858: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/wals/292aa245734c41d9814b8347d543b8fb
I20240501 19:56:35.587283 23168 catalog_manager.cc:4830] TS ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723): tablet 292aa245734c41d9814b8347d543b8fb (table rebalance_test_table_1 [id=25de88db313d414491562faffbac5131]) successfully deleted
I20240501 19:56:35.785259 23719 consensus_queue.cc:237] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 253, Committed index: 253, Last appended: 1.253, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:35.786717 23845 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 253. Preceding OpId from leader: term: 1 index: 254. (index mismatch)
W20240501 19:56:35.787045 23653 consensus_peers.cc:485] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 -> Peer d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8:35395): Couldn't send request to peer d6f0e2c51e3d4598b54766125f1d811f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5898d010acd94c35a73a6e5c28f8d313. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:35.787034 23326 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 253. Preceding OpId from leader: term: 1 index: 254. (index mismatch)
I20240501 19:56:35.787246 23457 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 253. Preceding OpId from leader: term: 1 index: 254. (index mismatch)
I20240501 19:56:35.787376 23583 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 253. Preceding OpId from leader: term: 1 index: 254. (index mismatch)
I20240501 19:56:35.787436 26322 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 254, Last known committed idx: 253, Time since last communication: 0.000s
I20240501 19:56:35.787721 26193 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 254, Last known committed idx: 253, Time since last communication: 0.000s
I20240501 19:56:35.787855 26255 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 254, Last known committed idx: 253, Time since last communication: 0.000s
I20240501 19:56:35.787957 26322 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 254, Last known committed idx: 253, Time since last communication: 0.000s
I20240501 19:56:35.789481 26255 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.254: config changed from index -1 to 254, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.790338 23583 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.254: config changed from index -1 to 254, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.790338 23326 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.254: config changed from index -1 to 254, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.791337 23845 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.254: config changed from index -1 to 254, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.793042 23179 catalog_manager.cc:5484] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 reported cstate change: config changed from index -1 to 254, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:35.794432 23457 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.254: config changed from index -1 to 254, NON_VOTER d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) added. New config: { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } }
I20240501 19:56:35.996835 26520 ts_tablet_manager.cc:902] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f: Initiating tablet copy from peer ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481)
I20240501 19:56:35.997272 26520 tablet_copy_client.cc:308] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:35481
I20240501 19:56:35.997663 23346 tablet_copy_service.cc:140] P ad9452840e4e4184a640fb63dab754fd: Received BeginTabletCopySession request for tablet a4f98ae45e914fa3981237af149b0666 from peer d6f0e2c51e3d4598b54766125f1d811f ({username='slave'} at 127.20.210.8:54995)
I20240501 19:56:35.997743 23346 tablet_copy_service.cc:161] P ad9452840e4e4184a640fb63dab754fd: Beginning new tablet copy session on tablet a4f98ae45e914fa3981237af149b0666 from peer d6f0e2c51e3d4598b54766125f1d811f at {username='slave'} at 127.20.210.8:54995: session id = d6f0e2c51e3d4598b54766125f1d811f-a4f98ae45e914fa3981237af149b0666
I20240501 19:56:35.998456 23346 tablet_copy_source_session.cc:205] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:35.999230 26520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4f98ae45e914fa3981237af149b0666. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:36.001057 26520 tablet_copy_client.cc:783] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:36.001194 26520 tablet_copy_client.cc:647] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:36.002446 26520 tablet_copy_client.cc:523] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:36.003283 26520 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap starting.
I20240501 19:56:36.021162 26527 ts_tablet_manager.cc:902] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f: Initiating tablet copy from peer 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:36.021778 26527 tablet_copy_client.cc:308] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:38367
I20240501 19:56:36.025980 23477 tablet_copy_service.cc:140] P 0496e037179e4b3ca72aead781c25d5a: Received BeginTabletCopySession request for tablet f701e0b045d540698ba3879954d8b171 from peer d6f0e2c51e3d4598b54766125f1d811f ({username='slave'} at 127.20.210.8:36867)
I20240501 19:56:36.026077 23477 tablet_copy_service.cc:161] P 0496e037179e4b3ca72aead781c25d5a: Beginning new tablet copy session on tablet f701e0b045d540698ba3879954d8b171 from peer d6f0e2c51e3d4598b54766125f1d811f at {username='slave'} at 127.20.210.8:36867: session id = d6f0e2c51e3d4598b54766125f1d811f-f701e0b045d540698ba3879954d8b171
I20240501 19:56:36.027395 23477 tablet_copy_source_session.cc:205] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:36.028143 26527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f701e0b045d540698ba3879954d8b171. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:36.029783 26527 tablet_copy_client.cc:783] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:36.029919 26527 tablet_copy_client.cc:647] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:36.032123 26527 tablet_copy_client.cc:523] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:36.033104 26527 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap starting.
I20240501 19:56:36.034197 26542 ts_tablet_manager.cc:902] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f: Initiating tablet copy from peer ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481)
I20240501 19:56:36.034714 26542 tablet_copy_client.cc:308] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:35481
I20240501 19:56:36.035143 23346 tablet_copy_service.cc:140] P ad9452840e4e4184a640fb63dab754fd: Received BeginTabletCopySession request for tablet b1a0cebcdad24e34abc11eef9d4d891d from peer d6f0e2c51e3d4598b54766125f1d811f ({username='slave'} at 127.20.210.8:54995)
I20240501 19:56:36.035248 23346 tablet_copy_service.cc:161] P ad9452840e4e4184a640fb63dab754fd: Beginning new tablet copy session on tablet b1a0cebcdad24e34abc11eef9d4d891d from peer d6f0e2c51e3d4598b54766125f1d811f at {username='slave'} at 127.20.210.8:54995: session id = d6f0e2c51e3d4598b54766125f1d811f-b1a0cebcdad24e34abc11eef9d4d891d
I20240501 19:56:36.036075 23346 tablet_copy_source_session.cc:205] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:36.036737 26542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1a0cebcdad24e34abc11eef9d4d891d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:36.038475 26542 tablet_copy_client.cc:783] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:36.038600 26542 tablet_copy_client.cc:647] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:36.040159 26542 tablet_copy_client.cc:523] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:36.041213 26542 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap starting.
I20240501 19:56:36.085714 26564 ts_tablet_manager.cc:902] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f: Initiating tablet copy from peer 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367)
I20240501 19:56:36.086196 26564 tablet_copy_client.cc:308] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:38367
I20240501 19:56:36.086645 23477 tablet_copy_service.cc:140] P 0496e037179e4b3ca72aead781c25d5a: Received BeginTabletCopySession request for tablet 87ebb3dfc00d4744960bde9a07c59e23 from peer d6f0e2c51e3d4598b54766125f1d811f ({username='slave'} at 127.20.210.8:36867)
I20240501 19:56:36.086731 23477 tablet_copy_service.cc:161] P 0496e037179e4b3ca72aead781c25d5a: Beginning new tablet copy session on tablet 87ebb3dfc00d4744960bde9a07c59e23 from peer d6f0e2c51e3d4598b54766125f1d811f at {username='slave'} at 127.20.210.8:36867: session id = d6f0e2c51e3d4598b54766125f1d811f-87ebb3dfc00d4744960bde9a07c59e23
I20240501 19:56:36.087579 23477 tablet_copy_source_session.cc:205] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:36.088414 26564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ebb3dfc00d4744960bde9a07c59e23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:36.093580 26564 tablet_copy_client.cc:783] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:36.093909 26564 tablet_copy_client.cc:647] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:36.098263 26564 tablet_copy_client.cc:523] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:36.099449 26564 tablet_bootstrap.cc:492] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap starting.
I20240501 19:56:36.176295 26520 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap replayed 1/1 log segments. Stats: ops{read=214 overwritten=0 applied=214 ignored=0} inserts{seen=896 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:36.176664 26520 tablet_bootstrap.cc:492] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap complete.
I20240501 19:56:36.176903 26520 ts_tablet_manager.cc:1367] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f: Time spent bootstrapping tablet: real 0.174s user 0.132s sys 0.013s
I20240501 19:56:36.177346 26520 raft_consensus.cc:348] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 214 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:36.177493 26520 raft_consensus.cc:729] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d6f0e2c51e3d4598b54766125f1d811f, State: Initialized, Role: LEARNER
I20240501 19:56:36.177603 26520 consensus_queue.cc:260] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 214, Last appended: 1.214, Last appended by leader: 214, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 214 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:36.177995 26520 ts_tablet_manager.cc:1398] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 19:56:36.180254 23346 tablet_copy_service.cc:342] P ad9452840e4e4184a640fb63dab754fd: Request end of tablet copy session d6f0e2c51e3d4598b54766125f1d811f-a4f98ae45e914fa3981237af149b0666 received from {username='slave'} at 127.20.210.8:54995
I20240501 19:56:36.180347 23346 tablet_copy_service.cc:433] P ad9452840e4e4184a640fb63dab754fd: ending tablet copy session d6f0e2c51e3d4598b54766125f1d811f-a4f98ae45e914fa3981237af149b0666 on tablet a4f98ae45e914fa3981237af149b0666 with peer d6f0e2c51e3d4598b54766125f1d811f
I20240501 19:56:36.181114 26520 ts_tablet_manager.cc:902] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f: Initiating tablet copy from peer 6e47806645ae47278e35600cc67d5700 (127.20.210.4:44657)
I20240501 19:56:36.181501 26520 tablet_copy_client.cc:308] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:44657
I20240501 19:56:36.181916 23739 tablet_copy_service.cc:140] P 6e47806645ae47278e35600cc67d5700: Received BeginTabletCopySession request for tablet 5898d010acd94c35a73a6e5c28f8d313 from peer d6f0e2c51e3d4598b54766125f1d811f ({username='slave'} at 127.20.210.8:48507)
I20240501 19:56:36.182022 23739 tablet_copy_service.cc:161] P 6e47806645ae47278e35600cc67d5700: Beginning new tablet copy session on tablet 5898d010acd94c35a73a6e5c28f8d313 from peer d6f0e2c51e3d4598b54766125f1d811f at {username='slave'} at 127.20.210.8:48507: session id = d6f0e2c51e3d4598b54766125f1d811f-5898d010acd94c35a73a6e5c28f8d313
I20240501 19:56:36.183004 23739 tablet_copy_source_session.cc:205] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:36.183815 26520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5898d010acd94c35a73a6e5c28f8d313. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:36.185824 26520 tablet_copy_client.cc:783] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:36.185971 26520 tablet_copy_client.cc:647] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:36.191831 26520 tablet_copy_client.cc:523] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:36.192910 26520 tablet_bootstrap.cc:492] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap starting.
I20240501 19:56:36.227002 26527 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap replayed 1/1 log segments. Stats: ops{read=259 overwritten=0 applied=259 ignored=0} inserts{seen=887 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:36.227381 26527 tablet_bootstrap.cc:492] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap complete.
I20240501 19:56:36.227558 26527 ts_tablet_manager.cc:1367] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f: Time spent bootstrapping tablet: real 0.194s user 0.163s sys 0.016s
I20240501 19:56:36.227916 26527 raft_consensus.cc:348] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 259 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:36.228031 26527 raft_consensus.cc:729] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d6f0e2c51e3d4598b54766125f1d811f, State: Initialized, Role: LEARNER
I20240501 19:56:36.228125 26527 consensus_queue.cc:260] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 259, Last appended: 1.259, Last appended by leader: 259, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 259 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:36.228538 26527 ts_tablet_manager.cc:1398] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:36.228900 23477 tablet_copy_service.cc:342] P 0496e037179e4b3ca72aead781c25d5a: Request end of tablet copy session d6f0e2c51e3d4598b54766125f1d811f-f701e0b045d540698ba3879954d8b171 received from {username='slave'} at 127.20.210.8:36867
I20240501 19:56:36.228981 23477 tablet_copy_service.cc:433] P 0496e037179e4b3ca72aead781c25d5a: ending tablet copy session d6f0e2c51e3d4598b54766125f1d811f-f701e0b045d540698ba3879954d8b171 on tablet f701e0b045d540698ba3879954d8b171 with peer d6f0e2c51e3d4598b54766125f1d811f
I20240501 19:56:36.238399 26542 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap replayed 1/1 log segments. Stats: ops{read=254 overwritten=0 applied=254 ignored=0} inserts{seen=936 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:36.238627 26542 tablet_bootstrap.cc:492] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap complete.
I20240501 19:56:36.238797 26542 ts_tablet_manager.cc:1367] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f: Time spent bootstrapping tablet: real 0.198s user 0.154s sys 0.026s
I20240501 19:56:36.239167 26542 raft_consensus.cc:348] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:36.239279 26542 raft_consensus.cc:729] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d6f0e2c51e3d4598b54766125f1d811f, State: Initialized, Role: LEARNER
I20240501 19:56:36.239364 26542 consensus_queue.cc:260] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 254, Last appended: 1.254, Last appended by leader: 254, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:36.239666 26542 ts_tablet_manager.cc:1398] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:36.240115 23346 tablet_copy_service.cc:342] P ad9452840e4e4184a640fb63dab754fd: Request end of tablet copy session d6f0e2c51e3d4598b54766125f1d811f-b1a0cebcdad24e34abc11eef9d4d891d received from {username='slave'} at 127.20.210.8:54995
I20240501 19:56:36.240195 23346 tablet_copy_service.cc:433] P ad9452840e4e4184a640fb63dab754fd: ending tablet copy session d6f0e2c51e3d4598b54766125f1d811f-b1a0cebcdad24e34abc11eef9d4d891d on tablet b1a0cebcdad24e34abc11eef9d4d891d with peer d6f0e2c51e3d4598b54766125f1d811f
I20240501 19:56:36.342619 26564 tablet_bootstrap.cc:492] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap replayed 1/1 log segments. Stats: ops{read=365 overwritten=0 applied=365 ignored=0} inserts{seen=996 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:36.342937 26564 tablet_bootstrap.cc:492] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap complete.
I20240501 19:56:36.343132 26564 ts_tablet_manager.cc:1367] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f: Time spent bootstrapping tablet: real 0.244s user 0.202s sys 0.028s
I20240501 19:56:36.343560 26564 raft_consensus.cc:348] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 365 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:36.343660 26564 raft_consensus.cc:729] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d6f0e2c51e3d4598b54766125f1d811f, State: Initialized, Role: LEARNER
I20240501 19:56:36.343771 26564 consensus_queue.cc:260] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 365, Last appended: 1.365, Last appended by leader: 365, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 365 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:36.344043 26564 ts_tablet_manager.cc:1398] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:36.344534 23477 tablet_copy_service.cc:342] P 0496e037179e4b3ca72aead781c25d5a: Request end of tablet copy session d6f0e2c51e3d4598b54766125f1d811f-87ebb3dfc00d4744960bde9a07c59e23 received from {username='slave'} at 127.20.210.8:36867
I20240501 19:56:36.344616 23477 tablet_copy_service.cc:433] P 0496e037179e4b3ca72aead781c25d5a: ending tablet copy session d6f0e2c51e3d4598b54766125f1d811f-87ebb3dfc00d4744960bde9a07c59e23 on tablet 87ebb3dfc00d4744960bde9a07c59e23 with peer d6f0e2c51e3d4598b54766125f1d811f
I20240501 19:56:36.365173 26520 tablet_bootstrap.cc:492] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap replayed 1/1 log segments. Stats: ops{read=254 overwritten=0 applied=254 ignored=0} inserts{seen=939 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:36.365473 26520 tablet_bootstrap.cc:492] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f: Bootstrap complete.
I20240501 19:56:36.365669 26520 ts_tablet_manager.cc:1367] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f: Time spent bootstrapping tablet: real 0.173s user 0.160s sys 0.000s
I20240501 19:56:36.366010 26520 raft_consensus.cc:348] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:36.366106 26520 raft_consensus.cc:729] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d6f0e2c51e3d4598b54766125f1d811f, State: Initialized, Role: LEARNER
I20240501 19:56:36.366196 26520 consensus_queue.cc:260] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 254, Last appended: 1.254, Last appended by leader: 254, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } }
I20240501 19:56:36.366505 26520 ts_tablet_manager.cc:1398] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 19:56:36.366984 23739 tablet_copy_service.cc:342] P 6e47806645ae47278e35600cc67d5700: Request end of tablet copy session d6f0e2c51e3d4598b54766125f1d811f-5898d010acd94c35a73a6e5c28f8d313 received from {username='slave'} at 127.20.210.8:48507
I20240501 19:56:36.367062 23739 tablet_copy_service.cc:433] P 6e47806645ae47278e35600cc67d5700: ending tablet copy session d6f0e2c51e3d4598b54766125f1d811f-5898d010acd94c35a73a6e5c28f8d313 on tablet 5898d010acd94c35a73a6e5c28f8d313 with peer d6f0e2c51e3d4598b54766125f1d811f
I20240501 19:56:36.427769 25883 raft_consensus.cc:1203] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.258->[1.259-1.259] Dedup: 1.259->[]
I20240501 19:56:36.501554 25883 raft_consensus.cc:1203] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.213->[1.214-1.214] Dedup: 1.214->[]
I20240501 19:56:36.509238 25883 raft_consensus.cc:1203] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.253->[1.254-1.254] Dedup: 1.254->[]
I20240501 19:56:36.569415 25883 raft_consensus.cc:1203] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.253->[1.254-1.254] Dedup: 1.254->[]
I20240501 19:56:36.590584 25883 raft_consensus.cc:1203] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.364->[1.365-1.365] Dedup: 1.365->[]
I20240501 19:56:36.929499 26331 raft_consensus.cc:1050] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd: attempting to promote NON_VOTER d6f0e2c51e3d4598b54766125f1d811f to VOTER
I20240501 19:56:36.929885 26331 consensus_queue.cc:237] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 214, Committed index: 214, Last appended: 1.214, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 215 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:36.931409 23849 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 214. Preceding OpId from leader: term: 1 index: 215. (index mismatch)
I20240501 19:56:36.931463 23719 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 214. Preceding OpId from leader: term: 1 index: 215. (index mismatch)
I20240501 19:56:36.931533 24755 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 214. Preceding OpId from leader: term: 1 index: 215. (index mismatch)
I20240501 19:56:36.931540 23583 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 214. Preceding OpId from leader: term: 1 index: 215. (index mismatch)
I20240501 19:56:36.931970 26658 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 215, Last known committed idx: 214, Time since last communication: 0.000s
I20240501 19:56:36.932091 25883 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 214. Preceding OpId from leader: term: 1 index: 215. (index mismatch)
I20240501 19:56:36.932152 26659 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 215, Last known committed idx: 214, Time since last communication: 0.000s
I20240501 19:56:36.932317 26657 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 215, Last known committed idx: 214, Time since last communication: 0.000s
I20240501 19:56:36.932446 26334 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 215, Last known committed idx: 214, Time since last communication: 0.000s
I20240501 19:56:36.932560 26331 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 215, Last known committed idx: 214, Time since last communication: 0.000s
I20240501 19:56:36.934772 26657 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.215: config changed from index 214 to 215, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 215 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:36.935163 24755 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.215: config changed from index 214 to 215, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 215 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:36.936682 23719 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.215: config changed from index 214 to 215, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 215 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:36.936936 23849 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.215: config changed from index 214 to 215, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 215 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:36.936975 23583 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.215: config changed from index 214 to 215, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 215 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:36.938386 25883 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.215: config changed from index 214 to 215, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 215 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:36.939316 23180 catalog_manager.cc:5484] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b reported cstate change: config changed from index 214 to 215, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 215 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:36.943084 23326 consensus_queue.cc:237] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 215, Committed index: 215, Last appended: 1.215, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 216 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:36.944531 25883 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 215. Preceding OpId from leader: term: 1 index: 216. (index mismatch)
I20240501 19:56:36.944777 24755 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 215. Preceding OpId from leader: term: 1 index: 216. (index mismatch)
I20240501 19:56:36.945065 23849 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 215. Preceding OpId from leader: term: 1 index: 216. (index mismatch)
I20240501 19:56:36.945158 26331 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 216, Last known committed idx: 215, Time since last communication: 0.000s
I20240501 19:56:36.945539 26604 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 216, Last known committed idx: 215, Time since last communication: 0.000s
I20240501 19:56:36.946763 26331 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 216, Last known committed idx: 215, Time since last communication: 0.000s
I20240501 19:56:36.946870 23719 raft_consensus.cc:1261] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 215. Preceding OpId from leader: term: 1 index: 216. (index mismatch)
I20240501 19:56:36.947160 26334 consensus_queue.cc:1035] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 216, Last known committed idx: 215, Time since last communication: 0.000s
I20240501 19:56:36.947095 26331 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.216: config changed from index 215 to 216, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 216 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:36.947710 23849 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Committing config change with OpId 1.216: config changed from index 215 to 216, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 216 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:36.948132 23719 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.216: config changed from index 215 to 216, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 216 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:36.949409 23170 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a4f98ae45e914fa3981237af149b0666 with cas_config_opid_index 215: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:36.949959 24755 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.216: config changed from index 215 to 216, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 216 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:36.950044 23184 catalog_manager.cc:5484] T a4f98ae45e914fa3981237af149b0666 P ee9bc706484e48fb8ff739a61cabe858 reported cstate change: config changed from index 215 to 216, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 216 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:36.950856 25883 raft_consensus.cc:2932] T a4f98ae45e914fa3981237af149b0666 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.216: config changed from index 215 to 216, VOTER e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3) evicted. New config: { opid_index: 216 OBSOLETE_local: false peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:36.952234 23568 tablet_service.cc:1514] Processing DeleteTablet for tablet a4f98ae45e914fa3981237af149b0666 with delete_type TABLET_DATA_TOMBSTONED (TS e8ddd74cfd954c7a8c2d284c335d7826 not found in new config with opid_index 216) from {username='slave'} at 127.0.0.1:46492
I20240501 19:56:36.952721 26670 tablet_replica.cc:330] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826: stopping tablet replica
I20240501 19:56:36.952947 26670 raft_consensus.cc:2229] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:36.953490 26670 raft_consensus.cc:2258] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:36.955224 26670 ts_tablet_manager.cc:1875] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:36.957021 26670 ts_tablet_manager.cc:1888] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.215
I20240501 19:56:36.957079 26670 log.cc:1196] T a4f98ae45e914fa3981237af149b0666 P e8ddd74cfd954c7a8c2d284c335d7826: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/a4f98ae45e914fa3981237af149b0666
I20240501 19:56:36.957504 23171 catalog_manager.cc:4830] TS e8ddd74cfd954c7a8c2d284c335d7826 (127.20.210.3:37279): tablet a4f98ae45e914fa3981237af149b0666 (table rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]) successfully deleted
I20240501 19:56:37.014027 26193 raft_consensus.cc:1050] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700: attempting to promote NON_VOTER d6f0e2c51e3d4598b54766125f1d811f to VOTER
I20240501 19:56:37.014534 26193 consensus_queue.cc:237] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 254, Committed index: 254, Last appended: 1.254, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:37.016026 25883 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 254. Preceding OpId from leader: term: 1 index: 255. (index mismatch)
I20240501 19:56:37.016333 23849 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 254. Preceding OpId from leader: term: 1 index: 255. (index mismatch)
I20240501 19:56:37.016458 23583 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 254. Preceding OpId from leader: term: 1 index: 255. (index mismatch)
I20240501 19:56:37.016714 23326 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 254. Preceding OpId from leader: term: 1 index: 255. (index mismatch)
I20240501 19:56:37.016752 23457 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 254. Preceding OpId from leader: term: 1 index: 255. (index mismatch)
I20240501 19:56:37.017053 26715 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 255, Last known committed idx: 254, Time since last communication: 0.000s
I20240501 19:56:37.017199 26712 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 255, Last known committed idx: 254, Time since last communication: 0.000s
I20240501 19:56:37.017297 26255 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 255, Last known committed idx: 254, Time since last communication: 0.000s
I20240501 19:56:37.017474 26713 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 255, Last known committed idx: 254, Time since last communication: 0.000s
I20240501 19:56:37.017832 26718 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 255, Last known committed idx: 254, Time since last communication: 0.000s
I20240501 19:56:37.019522 26603 raft_consensus.cc:1050] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a: attempting to promote NON_VOTER d6f0e2c51e3d4598b54766125f1d811f to VOTER
I20240501 19:56:37.019886 26603 consensus_queue.cc:237] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 259, Committed index: 259, Last appended: 1.259, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 260 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:37.020541 26713 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.255: config changed from index 254 to 255, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.021255 23583 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.255: config changed from index 254 to 255, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.022346 23719 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 259. Preceding OpId from leader: term: 1 index: 260. (index mismatch)
I20240501 19:56:37.022491 23581 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 259. Preceding OpId from leader: term: 1 index: 260. (index mismatch)
I20240501 19:56:37.022584 23457 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.255: config changed from index 254 to 255, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.022796 26526 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 260, Last known committed idx: 259, Time since last communication: 0.000s
I20240501 19:56:37.022946 24755 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 259. Preceding OpId from leader: term: 1 index: 260. (index mismatch)
I20240501 19:56:37.022933 25884 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.255: config changed from index 254 to 255, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.023895 25751 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 260, Last known committed idx: 259, Time since last communication: 0.000s
I20240501 19:56:37.024180 23326 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.255: config changed from index 254 to 255, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.024681 26726 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 260, Last known committed idx: 259, Time since last communication: 0.000s
I20240501 19:56:37.024976 23975 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 259. Preceding OpId from leader: term: 1 index: 260. (index mismatch)
I20240501 19:56:37.025048 25883 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 259. Preceding OpId from leader: term: 1 index: 260. (index mismatch)
I20240501 19:56:37.025278 23184 catalog_manager.cc:5484] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a reported cstate change: config changed from index 254 to 255, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.025728 26727 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 260, Last known committed idx: 259, Time since last communication: 0.000s
I20240501 19:56:37.025856 25751 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 260, Last known committed idx: 259, Time since last communication: 0.000s
I20240501 19:56:37.029917 23719 consensus_queue.cc:237] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 255, Committed index: 255, Last appended: 1.255, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:37.030798 26727 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Committing config change with OpId 1.260: config changed from index 259 to 260, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 260 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.031173 23457 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 255. Preceding OpId from leader: term: 1 index: 256. (index mismatch)
I20240501 19:56:37.031838 25884 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 255. Preceding OpId from leader: term: 1 index: 256. (index mismatch)
I20240501 19:56:37.031983 23326 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 255. Preceding OpId from leader: term: 1 index: 256. (index mismatch)
I20240501 19:56:37.032712 23583 raft_consensus.cc:1261] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 6e47806645ae47278e35600cc67d5700: Log matching property violated. Preceding OpId in replica: term: 1 index: 255. Preceding OpId from leader: term: 1 index: 256. (index mismatch)
I20240501 19:56:37.032696 26255 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 256, Last known committed idx: 255, Time since last communication: 0.000s
I20240501 19:56:37.032950 26255 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 256, Last known committed idx: 255, Time since last communication: 0.000s
I20240501 19:56:37.032891 23719 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.260: config changed from index 259 to 260, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 260 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.033066 26715 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 256, Last known committed idx: 255, Time since last communication: 0.000s
I20240501 19:56:37.033280 26255 consensus_queue.cc:1035] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 256, Last known committed idx: 255, Time since last communication: 0.000s
I20240501 19:56:37.034858 26193 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 [term 1 LEADER]: Committing config change with OpId 1.256: config changed from index 255 to 256, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.035182 23326 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.256: config changed from index 255 to 256, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.036703 23171 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5898d010acd94c35a73a6e5c28f8d313 with cas_config_opid_index 255: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:37.036849 23583 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.256: config changed from index 255 to 256, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.037048 23178 catalog_manager.cc:5484] T 5898d010acd94c35a73a6e5c28f8d313 P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index 255 to 256, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New cstate: current_term: 1 leader_uuid: "6e47806645ae47278e35600cc67d5700" committed_config { opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.037649 25884 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.256: config changed from index 255 to 256, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.039067 24755 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.260: config changed from index 259 to 260, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 260 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.039124 23177 catalog_manager.cc:5484] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a reported cstate change: config changed from index 259 to 260, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: 260 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:37.039896 23975 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.260: config changed from index 259 to 260, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 260 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.041543 23830 tablet_service.cc:1514] Processing DeleteTablet for tablet 5898d010acd94c35a73a6e5c28f8d313 with delete_type TABLET_DATA_TOMBSTONED (TS ee9bc706484e48fb8ff739a61cabe858 not found in new config with opid_index 256) from {username='slave'} at 127.0.0.1:57012
I20240501 19:56:37.042037 23457 raft_consensus.cc:2932] T 5898d010acd94c35a73a6e5c28f8d313 P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.256: config changed from index 255 to 256, VOTER ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5) evicted. New config: { opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.042557 23453 consensus_queue.cc:237] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 260, Committed index: 260, Last appended: 1.260, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:37.042614 23583 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.260: config changed from index 259 to 260, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 260 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.043874 25884 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.260: config changed from index 259 to 260, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 260 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false replace: true } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.044062 23974 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 260. Preceding OpId from leader: term: 1 index: 261. (index mismatch)
I20240501 19:56:37.044559 26526 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 261, Last known committed idx: 260, Time since last communication: 0.000s
I20240501 19:56:37.045220 23581 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 260. Preceding OpId from leader: term: 1 index: 261. (index mismatch)
I20240501 19:56:37.045377 26735 tablet_replica.cc:330] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858: stopping tablet replica
I20240501 19:56:37.045511 26735 raft_consensus.cc:2229] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:37.045639 26735 pending_rounds.cc:62] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858: Trying to abort 1 pending ops.
I20240501 19:56:37.045753 26526 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 261, Last known committed idx: 260, Time since last communication: 0.000s
I20240501 19:56:37.045742 26735 pending_rounds.cc:68] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858: Aborting op as it isn't in flight: id { term: 1 index: 255 } timestamp: 7022974554170126336 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "5898d010acd94c35a73a6e5c28f8d313" old_config { opid_index: 254 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: true } } } new_config { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ee9bc706484e48fb8ff739a61cabe858" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 44723 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } } }
I20240501 19:56:37.045857 26735 raft_consensus.cc:2894] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Aborting config change with OpId 1.255: Aborted: Op aborted
I20240501 19:56:37.045902 26735 raft_consensus.cc:2866] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240501 19:56:37.045929 26735 raft_consensus.cc:2258] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:37.048156 25884 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 260. Preceding OpId from leader: term: 1 index: 261. (index mismatch)
I20240501 19:56:37.048369 26735 ts_tablet_manager.cc:1875] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:37.048442 26526 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 261, Last known committed idx: 260, Time since last communication: 0.000s
I20240501 19:56:37.049252 23719 raft_consensus.cc:1261] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 260. Preceding OpId from leader: term: 1 index: 261. (index mismatch)
I20240501 19:56:37.049955 26728 consensus_queue.cc:1035] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 261, Last known committed idx: 260, Time since last communication: 0.000s
I20240501 19:56:37.050477 26735 ts_tablet_manager.cc:1888] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.255
I20240501 19:56:37.050577 26735 log.cc:1196] T 5898d010acd94c35a73a6e5c28f8d313 P ee9bc706484e48fb8ff739a61cabe858: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/wals/5898d010acd94c35a73a6e5c28f8d313
I20240501 19:56:37.050477 26526 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Committing config change with OpId 1.261: config changed from index 260 to 261, VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) evicted. New config: { opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.050895 25884 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.261: config changed from index 260 to 261, VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) evicted. New config: { opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.051766 23168 catalog_manager.cc:4830] TS ee9bc706484e48fb8ff739a61cabe858 (127.20.210.5:44723): tablet 5898d010acd94c35a73a6e5c28f8d313 (table rebalance_test_table_2 [id=510250fac81b4e55abfe84644afce823]) successfully deleted
I20240501 19:56:37.052510 23581 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.261: config changed from index 260 to 261, VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) evicted. New config: { opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.052955 23168 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f701e0b045d540698ba3879954d8b171 with cas_config_opid_index 260: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:37.053558 23974 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.261: config changed from index 260 to 261, VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) evicted. New config: { opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.053682 23178 catalog_manager.cc:5484] T f701e0b045d540698ba3879954d8b171 P 0496e037179e4b3ca72aead781c25d5a reported cstate change: config changed from index 260 to 261, VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) evicted. New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:37.056046 23719 raft_consensus.cc:2932] T f701e0b045d540698ba3879954d8b171 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.261: config changed from index 260 to 261, VOTER 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7) evicted. New config: { opid_index: 261 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.059990 24737 tablet_service.cc:1514] Processing DeleteTablet for tablet f701e0b045d540698ba3879954d8b171 with delete_type TABLET_DATA_TOMBSTONED (TS 3b3f502cbeb1451087a06f95a3d93f2b not found in new config with opid_index 261) from {username='slave'} at 127.0.0.1:40496
I20240501 19:56:37.060379 26739 tablet_replica.cc:330] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: stopping tablet replica
I20240501 19:56:37.060621 26739 raft_consensus.cc:2229] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:37.060818 26739 raft_consensus.cc:2258] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:37.062207 26739 ts_tablet_manager.cc:1875] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:37.063812 26739 ts_tablet_manager.cc:1888] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.260
I20240501 19:56:37.063901 26739 log.cc:1196] T f701e0b045d540698ba3879954d8b171 P 3b3f502cbeb1451087a06f95a3d93f2b: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal/wals/f701e0b045d540698ba3879954d8b171
I20240501 19:56:37.064368 23169 catalog_manager.cc:4830] TS 3b3f502cbeb1451087a06f95a3d93f2b (127.20.210.7:35019): tablet f701e0b045d540698ba3879954d8b171 (table rebalance_test_table_4 [id=e43b7dd9423e43ddbb5b05c39ca91ce3]) successfully deleted
I20240501 19:56:37.080765 26728 raft_consensus.cc:1050] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a: attempting to promote NON_VOTER d6f0e2c51e3d4598b54766125f1d811f to VOTER
I20240501 19:56:37.081090 26728 consensus_queue.cc:237] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 365, Committed index: 365, Last appended: 1.365, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 366 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:37.082428 25884 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 365. Preceding OpId from leader: term: 1 index: 366. (index mismatch)
I20240501 19:56:37.082428 23326 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 365. Preceding OpId from leader: term: 1 index: 366. (index mismatch)
I20240501 19:56:37.082664 23974 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 365. Preceding OpId from leader: term: 1 index: 366. (index mismatch)
I20240501 19:56:37.082743 23719 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 365. Preceding OpId from leader: term: 1 index: 366. (index mismatch)
I20240501 19:56:37.082729 26603 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 366, Last known committed idx: 365, Time since last communication: 0.000s
I20240501 19:56:37.082947 23581 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 365. Preceding OpId from leader: term: 1 index: 366. (index mismatch)
I20240501 19:56:37.083091 26728 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 366, Last known committed idx: 365, Time since last communication: 0.000s
I20240501 19:56:37.083262 25751 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 366, Last known committed idx: 365, Time since last communication: 0.000s
I20240501 19:56:37.083523 26526 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 366, Last known committed idx: 365, Time since last communication: 0.000s
I20240501 19:56:37.083667 26603 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 366, Last known committed idx: 365, Time since last communication: 0.000s
I20240501 19:56:37.085342 26526 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Committing config change with OpId 1.366: config changed from index 365 to 366, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 366 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.086025 23581 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.366: config changed from index 365 to 366, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 366 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.086339 25884 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.366: config changed from index 365 to 366, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 366 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.086381 23719 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.366: config changed from index 365 to 366, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 366 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.088070 23326 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.366: config changed from index 365 to 366, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 366 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.088603 23182 catalog_manager.cc:5484] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 reported cstate change: config changed from index 365 to 366, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: 366 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.089229 23974 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.366: config changed from index 365 to 366, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 366 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } attrs { replace: true } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.092700 23457 consensus_queue.cc:237] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 366, Committed index: 366, Last appended: 1.366, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 367 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:37.093896 23326 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 366. Preceding OpId from leader: term: 1 index: 367. (index mismatch)
I20240501 19:56:37.093950 23719 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 366. Preceding OpId from leader: term: 1 index: 367. (index mismatch)
I20240501 19:56:37.094182 25884 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 366. Preceding OpId from leader: term: 1 index: 367. (index mismatch)
I20240501 19:56:37.094286 23581 raft_consensus.cc:1261] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Refusing update from remote peer 0496e037179e4b3ca72aead781c25d5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 366. Preceding OpId from leader: term: 1 index: 367. (index mismatch)
I20240501 19:56:37.094308 26727 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 367, Last known committed idx: 366, Time since last communication: 0.000s
I20240501 19:56:37.094591 26603 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 367, Last known committed idx: 366, Time since last communication: 0.000s
I20240501 19:56:37.095111 26727 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 367, Last known committed idx: 366, Time since last communication: 0.000s
I20240501 19:56:37.095283 26603 consensus_queue.cc:1035] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 367, Last known committed idx: 366, Time since last communication: 0.000s
I20240501 19:56:37.096297 26603 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a [term 1 LEADER]: Committing config change with OpId 1.367: config changed from index 366 to 367, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 367 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.097312 23719 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.367: config changed from index 366 to 367, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 367 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.098008 23326 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P ad9452840e4e4184a640fb63dab754fd [term 1 FOLLOWER]: Committing config change with OpId 1.367: config changed from index 366 to 367, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 367 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.098424 23168 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 87ebb3dfc00d4744960bde9a07c59e23 with cas_config_opid_index 366: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:37.099112 23581 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P e8ddd74cfd954c7a8c2d284c335d7826 [term 1 FOLLOWER]: Committing config change with OpId 1.367: config changed from index 366 to 367, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 367 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.099223 23177 catalog_manager.cc:5484] T 87ebb3dfc00d4744960bde9a07c59e23 P 0496e037179e4b3ca72aead781c25d5a reported cstate change: config changed from index 366 to 367, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New cstate: current_term: 1 leader_uuid: "0496e037179e4b3ca72aead781c25d5a" committed_config { opid_index: 367 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:37.100049 25884 raft_consensus.cc:2932] T 87ebb3dfc00d4744960bde9a07c59e23 P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.367: config changed from index 366 to 367, VOTER 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6) evicted. New config: { opid_index: 367 OBSOLETE_local: false peers { permanent_uuid: "e8ddd74cfd954c7a8c2d284c335d7826" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 37279 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.101594 23961 tablet_service.cc:1514] Processing DeleteTablet for tablet 87ebb3dfc00d4744960bde9a07c59e23 with delete_type TABLET_DATA_TOMBSTONED (TS 4f51f982ad284c22aae01c4541a621f1 not found in new config with opid_index 367) from {username='slave'} at 127.0.0.1:45934
I20240501 19:56:37.101938 26746 tablet_replica.cc:330] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1: stopping tablet replica
I20240501 19:56:37.102221 26746 raft_consensus.cc:2229] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:37.102479 26746 raft_consensus.cc:2258] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:37.104074 26746 ts_tablet_manager.cc:1875] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:37.105840 26746 ts_tablet_manager.cc:1888] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.366
I20240501 19:56:37.105904 26746 log.cc:1196] T 87ebb3dfc00d4744960bde9a07c59e23 P 4f51f982ad284c22aae01c4541a621f1: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/wals/87ebb3dfc00d4744960bde9a07c59e23
I20240501 19:56:37.106388 23169 catalog_manager.cc:4830] TS 4f51f982ad284c22aae01c4541a621f1 (127.20.210.6:37667): tablet 87ebb3dfc00d4744960bde9a07c59e23 (table rebalance_test_table_0 [id=ffcc6e469144432faca7b1c39980bdb7]) successfully deleted
I20240501 19:56:37.128124 26659 raft_consensus.cc:1050] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd: attempting to promote NON_VOTER d6f0e2c51e3d4598b54766125f1d811f to VOTER
I20240501 19:56:37.128657 26659 consensus_queue.cc:237] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 254, Committed index: 254, Last appended: 1.254, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:37.130488 23719 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 254. Preceding OpId from leader: term: 1 index: 255. (index mismatch)
I20240501 19:56:37.130506 23974 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 254. Preceding OpId from leader: term: 1 index: 255. (index mismatch)
I20240501 19:56:37.130564 23457 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 254. Preceding OpId from leader: term: 1 index: 255. (index mismatch)
I20240501 19:56:37.130925 25884 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f [term 1 LEARNER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 254. Preceding OpId from leader: term: 1 index: 255. (index mismatch)
I20240501 19:56:37.131016 24755 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 254. Preceding OpId from leader: term: 1 index: 255. (index mismatch)
I20240501 19:56:37.131163 26657 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 255, Last known committed idx: 254, Time since last communication: 0.000s
I20240501 19:56:37.131347 26659 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 255, Last known committed idx: 254, Time since last communication: 0.000s
I20240501 19:56:37.131501 26331 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 255, Last known committed idx: 254, Time since last communication: 0.000s
I20240501 19:56:37.131804 26331 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 255, Last known committed idx: 254, Time since last communication: 0.000s
I20240501 19:56:37.132597 26334 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 255, Last known committed idx: 254, Time since last communication: 0.000s
I20240501 19:56:37.134282 26334 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.255: config changed from index 254 to 255, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.134797 25884 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.255: config changed from index 254 to 255, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.135290 23457 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Committing config change with OpId 1.255: config changed from index 254 to 255, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.135490 23974 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.255: config changed from index 254 to 255, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.136884 23719 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.255: config changed from index 254 to 255, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.138610 23177 catalog_manager.cc:5484] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f reported cstate change: config changed from index 254 to 255, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.139415 24755 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.255: config changed from index 254 to 255, d6f0e2c51e3d4598b54766125f1d811f (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 255 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "0496e037179e4b3ca72aead781c25d5a" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 38367 } attrs { replace: true } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.142954 23326 consensus_queue.cc:237] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 255, Committed index: 255, Last appended: 1.255, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } }
I20240501 19:56:37.144287 23719 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 255. Preceding OpId from leader: term: 1 index: 256. (index mismatch)
I20240501 19:56:37.144521 24755 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 255. Preceding OpId from leader: term: 1 index: 256. (index mismatch)
I20240501 19:56:37.144855 26657 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 256, Last known committed idx: 255, Time since last communication: 0.000s
I20240501 19:56:37.144982 23974 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 255. Preceding OpId from leader: term: 1 index: 256. (index mismatch)
I20240501 19:56:37.145020 26331 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 256, Last known committed idx: 255, Time since last communication: 0.000s
I20240501 19:56:37.145287 26658 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 256, Last known committed idx: 255, Time since last communication: 0.000s
I20240501 19:56:37.146504 25884 raft_consensus.cc:1261] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Refusing update from remote peer ad9452840e4e4184a640fb63dab754fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 255. Preceding OpId from leader: term: 1 index: 256. (index mismatch)
I20240501 19:56:37.147138 26657 consensus_queue.cc:1035] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 256, Last known committed idx: 255, Time since last communication: 0.000s
I20240501 19:56:37.147686 26658 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd [term 1 LEADER]: Committing config change with OpId 1.256: config changed from index 255 to 256, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.148145 23974 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 4f51f982ad284c22aae01c4541a621f1 [term 1 FOLLOWER]: Committing config change with OpId 1.256: config changed from index 255 to 256, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.149592 23170 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b1a0cebcdad24e34abc11eef9d4d891d with cas_config_opid_index 255: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:37.149605 23719 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 6e47806645ae47278e35600cc67d5700 [term 1 FOLLOWER]: Committing config change with OpId 1.256: config changed from index 255 to 256, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.150493 23177 catalog_manager.cc:5484] T b1a0cebcdad24e34abc11eef9d4d891d P ad9452840e4e4184a640fb63dab754fd reported cstate change: config changed from index 255 to 256, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "ad9452840e4e4184a640fb63dab754fd" committed_config { opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:37.153416 23435 tablet_service.cc:1514] Processing DeleteTablet for tablet b1a0cebcdad24e34abc11eef9d4d891d with delete_type TABLET_DATA_TOMBSTONED (TS 0496e037179e4b3ca72aead781c25d5a not found in new config with opid_index 256) from {username='slave'} at 127.0.0.1:49870
I20240501 19:56:37.153951 26753 tablet_replica.cc:330] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a: stopping tablet replica
I20240501 19:56:37.154129 26753 raft_consensus.cc:2229] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:37.154341 26753 raft_consensus.cc:2258] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:37.157044 26753 ts_tablet_manager.cc:1875] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:37.156890 25884 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P d6f0e2c51e3d4598b54766125f1d811f [term 1 FOLLOWER]: Committing config change with OpId 1.256: config changed from index 255 to 256, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.159931 26753 ts_tablet_manager.cc:1888] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.255
I20240501 19:56:37.160045 26753 log.cc:1196] T b1a0cebcdad24e34abc11eef9d4d891d P 0496e037179e4b3ca72aead781c25d5a: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/b1a0cebcdad24e34abc11eef9d4d891d
I20240501 19:56:37.160061 24755 raft_consensus.cc:2932] T b1a0cebcdad24e34abc11eef9d4d891d P 3b3f502cbeb1451087a06f95a3d93f2b [term 1 FOLLOWER]: Committing config change with OpId 1.256: config changed from index 255 to 256, VOTER 0496e037179e4b3ca72aead781c25d5a (127.20.210.2) evicted. New config: { opid_index: 256 OBSOLETE_local: false peers { permanent_uuid: "4f51f982ad284c22aae01c4541a621f1" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 37667 } } peers { permanent_uuid: "ad9452840e4e4184a640fb63dab754fd" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 35481 } } peers { permanent_uuid: "6e47806645ae47278e35600cc67d5700" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 44657 } } peers { permanent_uuid: "3b3f502cbeb1451087a06f95a3d93f2b" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 35019 } attrs { promote: false } } peers { permanent_uuid: "d6f0e2c51e3d4598b54766125f1d811f" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 35395 } attrs { promote: false } } }
I20240501 19:56:37.161362 23168 catalog_manager.cc:4830] TS 0496e037179e4b3ca72aead781c25d5a (127.20.210.2:38367): tablet b1a0cebcdad24e34abc11eef9d4d891d (table rebalance_test_table_3 [id=40a9a76ff0f04080a591a2a11c721af1]) successfully deleted
I20240501 19:56:37.301571 23698 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:37.302384 23435 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:37.304734 24737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:37.310847 25864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:37.312791 23305 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:37.317052 23568 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:37.322444 23830 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:37.324649 23961 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:37.414005 23435 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:37.416662 23698 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:37.419077 24737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:37.419940 23305 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:37.423079 23568 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:37.429560 23961 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:37.435607 23830 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:37.435662 25864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:37.539271 23435 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:37.539458 23961 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:37.539721 23698 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:37.552369 23305 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:56:37.552623 24737 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:37.554643 25864 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:37.566932 23830 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:56:37.579066 23568 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
bb28524133a341a69c1f36bc25de2d8d | 127.20.210.62:37935 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.20.210.20:35945 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
0496e037179e4b3ca72aead781c25d5a | 127.20.210.2:38367 | HEALTHY | <none> | 3 | 0
3b3f502cbeb1451087a06f95a3d93f2b | 127.20.210.7:35019 | HEALTHY | <none> | 0 | 0
4f51f982ad284c22aae01c4541a621f1 | 127.20.210.6:37667 | HEALTHY | <none> | 1 | 0
6e47806645ae47278e35600cc67d5700 | 127.20.210.4:44657 | HEALTHY | <none> | 4 | 0
ad9452840e4e4184a640fb63dab754fd | 127.20.210.1:35481 | HEALTHY | <none> | 3 | 0
d6f0e2c51e3d4598b54766125f1d811f | 127.20.210.8:35395 | HEALTHY | <none> | 0 | 0
e8ddd74cfd954c7a8c2d284c335d7826 | 127.20.210.3:37279 | HEALTHY | <none> | 1 | 0
ee9bc706484e48fb8ff739a61cabe858 | 127.20.210.5:44723 | HEALTHY | <none> | 4 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 8
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 8 server(s) checked
local_ip_for_outbound_sockets | 127.20.210.1 | experimental | 127.20.210.1:35481
local_ip_for_outbound_sockets | 127.20.210.2 | experimental | 127.20.210.2:38367
local_ip_for_outbound_sockets | 127.20.210.3 | experimental | 127.20.210.3:37279
local_ip_for_outbound_sockets | 127.20.210.4 | experimental | 127.20.210.4:44657
local_ip_for_outbound_sockets | 127.20.210.5 | experimental | 127.20.210.5:44723
local_ip_for_outbound_sockets | 127.20.210.6 | experimental | 127.20.210.6:37667
local_ip_for_outbound_sockets | 127.20.210.7 | experimental | 127.20.210.7:35019
local_ip_for_outbound_sockets | 127.20.210.8 | experimental | 127.20.210.8:35395
never_fsync | true | unsafe,advanced | all 8 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 8 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 8 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 8 server(s) checked
server_dump_info_format | pb | hidden | all 8 server(s) checked
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.20.210.1:35481
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.20.210.2:38367
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.20.210.3:37279
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.20.210.4:44657
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.20.210.5:44723
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.20.210.6:37667
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.20.210.7:35019
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.20.210.8:35395
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.20.210.20:35945 | all 8 server(s) checked
time_source | builtin | all 8 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 9 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 5 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 10
First Quartile | 10
Median | 10
Third Quartile | 10
Maximum | 10
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 8
Tables | 6
Tablets | 16
Replicas | 80
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240501 19:56:37.601079 21320 log_verifier.cc:126] Checking tablet 107c33af3fbf4edbb3d3b4ba880e4661
I20240501 19:56:37.738978 21320 log_verifier.cc:177] Verified matching terms for 359 ops in tablet 107c33af3fbf4edbb3d3b4ba880e4661
I20240501 19:56:37.739177 21320 log_verifier.cc:126] Checking tablet 1c9623f516f84445ade4bf06eb05673f
I20240501 19:56:37.845983 21320 log_verifier.cc:177] Verified matching terms for 266 ops in tablet 1c9623f516f84445ade4bf06eb05673f
I20240501 19:56:37.846179 21320 log_verifier.cc:126] Checking tablet 292aa245734c41d9814b8347d543b8fb
I20240501 19:56:37.982043 21320 log_verifier.cc:177] Verified matching terms for 343 ops in tablet 292aa245734c41d9814b8347d543b8fb
I20240501 19:56:37.982280 21320 log_verifier.cc:126] Checking tablet 44f96d2d427d43008630eb7bcc39caf9
I20240501 19:56:38.008535 21320 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 44f96d2d427d43008630eb7bcc39caf9
I20240501 19:56:38.008591 21320 log_verifier.cc:126] Checking tablet 5898d010acd94c35a73a6e5c28f8d313
I20240501 19:56:38.119997 21320 log_verifier.cc:177] Verified matching terms for 256 ops in tablet 5898d010acd94c35a73a6e5c28f8d313
I20240501 19:56:38.120172 21320 log_verifier.cc:126] Checking tablet 6f549fed054d433381ac6d38d64c74bb
I20240501 19:56:38.238426 21320 log_verifier.cc:177] Verified matching terms for 263 ops in tablet 6f549fed054d433381ac6d38d64c74bb
I20240501 19:56:38.238612 21320 log_verifier.cc:126] Checking tablet 78b919c4db7745fd87151d8834866e26
I20240501 19:56:38.391455 21320 log_verifier.cc:177] Verified matching terms for 397 ops in tablet 78b919c4db7745fd87151d8834866e26
I20240501 19:56:38.391759 21320 log_verifier.cc:126] Checking tablet 87ebb3dfc00d4744960bde9a07c59e23
I20240501 19:56:38.532312 21320 log_verifier.cc:177] Verified matching terms for 367 ops in tablet 87ebb3dfc00d4744960bde9a07c59e23
I20240501 19:56:38.532491 21320 log_verifier.cc:126] Checking tablet 9f0625a730b0408b963d94f24e25fed7
I20240501 19:56:38.639259 21320 log_verifier.cc:177] Verified matching terms for 258 ops in tablet 9f0625a730b0408b963d94f24e25fed7
I20240501 19:56:38.639439 21320 log_verifier.cc:126] Checking tablet a4ef2fdbb40e48b18025136adb39defc
I20240501 19:56:38.737617 21320 log_verifier.cc:177] Verified matching terms for 236 ops in tablet a4ef2fdbb40e48b18025136adb39defc
I20240501 19:56:38.737751 21320 log_verifier.cc:126] Checking tablet a4f98ae45e914fa3981237af149b0666
I20240501 19:56:38.831827 21320 log_verifier.cc:177] Verified matching terms for 216 ops in tablet a4f98ae45e914fa3981237af149b0666
I20240501 19:56:38.831969 21320 log_verifier.cc:126] Checking tablet af8e2ce72fb24a18b54b0d2e49383b41
I20240501 19:56:38.968811 21320 log_verifier.cc:177] Verified matching terms for 366 ops in tablet af8e2ce72fb24a18b54b0d2e49383b41
I20240501 19:56:38.969002 21320 log_verifier.cc:126] Checking tablet b1a0cebcdad24e34abc11eef9d4d891d
I20240501 19:56:39.078521 21320 log_verifier.cc:177] Verified matching terms for 256 ops in tablet b1a0cebcdad24e34abc11eef9d4d891d
I20240501 19:56:39.078714 21320 log_verifier.cc:126] Checking tablet b2eea4673ba94b0fa5a2e0f099532777
I20240501 19:56:39.233300 21320 log_verifier.cc:177] Verified matching terms for 367 ops in tablet b2eea4673ba94b0fa5a2e0f099532777
I20240501 19:56:39.233639 21320 log_verifier.cc:126] Checking tablet f53fd7f6fb694eb89eb611497e064ab5
I20240501 19:56:39.335582 21320 log_verifier.cc:177] Verified matching terms for 240 ops in tablet f53fd7f6fb694eb89eb611497e064ab5
I20240501 19:56:39.335742 21320 log_verifier.cc:126] Checking tablet f701e0b045d540698ba3879954d8b171
I20240501 19:56:39.447306 21320 log_verifier.cc:177] Verified matching terms for 261 ops in tablet f701e0b045d540698ba3879954d8b171
I20240501 19:56:39.448843 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 23244
W20240501 19:56:39.457217 23654 connection.cc:535] client connection to 127.20.210.1:35481 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240501 19:56:39.457361 23654 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240501 19:56:39.457917 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 23375
W20240501 19:56:39.459543 23654 consensus_peers.cc:485] T 5898d010acd94c35a73a6e5c28f8d313 P 6e47806645ae47278e35600cc67d5700 -> Peer ad9452840e4e4184a640fb63dab754fd (127.20.210.1:35481): Couldn't send request to peer ad9452840e4e4184a640fb63dab754fd. Status: Network error: Client connection negotiation failed: client connection to 127.20.210.1:35481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:39.465056 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 23506
I20240501 19:56:39.472844 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 23637
I20240501 19:56:39.479586 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 23768
I20240501 19:56:39.485821 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 23899
I20240501 19:56:39.491834 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 24675
I20240501 19:56:39.497414 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 25802
I20240501 19:56:39.502521 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 23153
2024-05-01T19:56:39Z chronyd exiting
[ OK ] RebalanceParamTest.Rebalance/7 (33168 ms)
[----------] 1 test from RebalanceParamTest (33168 ms total)
[----------] 1 test from TserverAddedDuringRebalancingTest
[ RUN ] TserverAddedDuringRebalancingTest.TserverStarts/1
I20240501 19:56:39.595964 21320 test_util.cc:268] Using random seed: -494640203
I20240501 19:56:39.596688 21320 ts_itest-base.cc:112] Starting cluster with:
I20240501 19:56:39.596735 21320 ts_itest-base.cc:113] --------------
I20240501 19:56:39.596745 21320 ts_itest-base.cc:114] 8 tablet servers
I20240501 19:56:39.596755 21320 ts_itest-base.cc:115] 3 replicas per TS
I20240501 19:56:39.596766 21320 ts_itest-base.cc:116] --------------
2024-05-01T19:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T19:56:39Z Disabled control of system clock
I20240501 19:56:39.604954 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:43635
--webserver_interface=127.20.210.62
--webserver_port=0
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.210.62:43635
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240501 19:56:39.702988 26893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:39.703187 26893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:39.703226 26893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:39.706131 26893 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:39.706190 26893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 19:56:39.706229 26893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:39.706281 26893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 19:56:39.706317 26893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 19:56:39.709661 26893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.20.210.62:43635
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:43635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.20.210.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.26893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:39.710443 26893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:39.711433 26893 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:39.718703 26898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:39.719419 26893 server_base.cc:1034] running on GCE node
W20240501 19:56:39.720045 26901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:39.720710 26899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:39.721199 26893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:39.722002 26893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:39.723145 26893 hybrid_clock.cc:648] HybridClock initialized: now 1714593399723121 us; error 44 us; skew 500 ppm
I20240501 19:56:39.728822 26893 webserver.cc:460] Webserver started at http://127.20.210.62:33695/ using document root <none> and password file <none>
I20240501 19:56:39.729321 26893 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:39.729382 26893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:39.729537 26893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:39.731024 26893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "8496c8b9092949be83f5731019951f7e"
format_stamp: "Formatted at 2024-05-01 19:56:39 on dist-test-slave-vgh4"
I20240501 19:56:39.731463 26893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "8496c8b9092949be83f5731019951f7e"
format_stamp: "Formatted at 2024-05-01 19:56:39 on dist-test-slave-vgh4"
I20240501 19:56:39.734407 26893 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 19:56:39.736387 26907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:39.737306 26893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:39.737412 26893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
uuid: "8496c8b9092949be83f5731019951f7e"
format_stamp: "Formatted at 2024-05-01 19:56:39 on dist-test-slave-vgh4"
I20240501 19:56:39.737488 26893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:39.758761 26893 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:39.759311 26893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:39.759452 26893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:39.765630 26893 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.62:43635
I20240501 19:56:39.765632 26959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.62:43635 every 8 connection(s)
I20240501 19:56:39.766549 26893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 19:56:39.768939 26960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:39.769503 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 26893
I20240501 19:56:39.769618 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 19:56:39.773638 26960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e: Bootstrap starting.
I20240501 19:56:39.775550 26960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:39.776213 26960 log.cc:824] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:39.777619 26960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e: No bootstrap required, opened a new log
I20240501 19:56:39.779878 26960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8496c8b9092949be83f5731019951f7e" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 43635 } }
I20240501 19:56:39.780032 26960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:39.780081 26960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8496c8b9092949be83f5731019951f7e, State: Initialized, Role: FOLLOWER
I20240501 19:56:39.780418 26960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8496c8b9092949be83f5731019951f7e" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 43635 } }
I20240501 19:56:39.780529 26960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:56:39.780591 26960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:56:39.780694 26960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:39.781291 26960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8496c8b9092949be83f5731019951f7e" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 43635 } }
I20240501 19:56:39.781563 26960 leader_election.cc:302] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8496c8b9092949be83f5731019951f7e; no voters:
I20240501 19:56:39.781826 26960 leader_election.cc:288] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:56:39.782122 26965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:39.782335 26965 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [term 1 LEADER]: Becoming Leader. State: Replica: 8496c8b9092949be83f5731019951f7e, State: Running, Role: LEADER
I20240501 19:56:39.782604 26965 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8496c8b9092949be83f5731019951f7e" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 43635 } }
I20240501 19:56:39.782814 26960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [sys.catalog]: configured and running, proceeding with master startup.
I20240501 19:56:39.783885 26966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8496c8b9092949be83f5731019951f7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8496c8b9092949be83f5731019951f7e" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 43635 } } }
I20240501 19:56:39.783986 26966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [sys.catalog]: This master's current role is: LEADER
I20240501 19:56:39.784301 26967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8496c8b9092949be83f5731019951f7e. Latest consensus state: current_term: 1 leader_uuid: "8496c8b9092949be83f5731019951f7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8496c8b9092949be83f5731019951f7e" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 43635 } } }
I20240501 19:56:39.784359 26974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 19:56:39.784425 26967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e [sys.catalog]: This master's current role is: LEADER
I20240501 19:56:39.786849 26974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 19:56:39.791090 26974 catalog_manager.cc:1347] Generated new cluster ID: e62d2eaa050e47d1810ddc3726edcf70
I20240501 19:56:39.791150 26974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 19:56:39.800478 26974 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 19:56:39.801216 26974 catalog_manager.cc:1504] Loading token signing keys...
I20240501 19:56:39.809233 26974 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 8496c8b9092949be83f5731019951f7e: Generated new TSK 0
I20240501 19:56:39.809799 26974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 19:56:39.814694 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.1:0
--local_ip_for_outbound_sockets=127.20.210.1
--webserver_interface=127.20.210.1
--webserver_port=0
--tserver_master_addrs=127.20.210.62:43635
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240501 19:56:39.913348 26984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:39.913519 26984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:39.913564 26984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:39.916450 26984 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:39.916502 26984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:39.916581 26984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.1
I20240501 19:56:39.919975 26984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.20.210.1
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.20.210.62:43635
--never_fsync=true
--heap_profile_path=/tmp/kudu.26984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.1
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:39.920711 26984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:39.921741 26984 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:39.929710 26989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:39.930367 26990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:39.931042 26992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:39.931800 26984 server_base.cc:1034] running on GCE node
I20240501 19:56:39.932092 26984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:39.932550 26984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:39.933706 26984 hybrid_clock.cc:648] HybridClock initialized: now 1714593399933691 us; error 31 us; skew 500 ppm
I20240501 19:56:39.935487 26984 webserver.cc:460] Webserver started at http://127.20.210.1:40607/ using document root <none> and password file <none>
I20240501 19:56:39.935981 26984 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:39.936029 26984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:39.936173 26984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:39.937527 26984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
format_stamp: "Formatted at 2024-05-01 19:56:39 on dist-test-slave-vgh4"
I20240501 19:56:39.937932 26984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
format_stamp: "Formatted at 2024-05-01 19:56:39 on dist-test-slave-vgh4"
I20240501 19:56:39.940596 26984 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20240501 19:56:39.942428 26998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:39.943243 26984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:39.943326 26984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
format_stamp: "Formatted at 2024-05-01 19:56:39 on dist-test-slave-vgh4"
I20240501 19:56:39.943401 26984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:39.959604 26984 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:39.960031 26984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:39.960157 26984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:39.960578 26984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:39.961254 26984 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:39.961299 26984 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:39.961351 26984 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:39.961380 26984 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:39.968835 26984 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.1:33375
I20240501 19:56:39.968865 27111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.1:33375 every 8 connection(s)
I20240501 19:56:39.969566 26984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 19:56:39.969735 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 26984
I20240501 19:56:39.969841 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 19:56:39.971984 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.2:0
--local_ip_for_outbound_sockets=127.20.210.2
--webserver_interface=127.20.210.2
--webserver_port=0
--tserver_master_addrs=127.20.210.62:43635
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 19:56:39.977995 27112 heartbeater.cc:347] Connected to a master server at 127.20.210.62:43635
I20240501 19:56:39.978204 27112 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:39.978788 27112 heartbeater.cc:511] Master 127.20.210.62:43635 requested a full tablet report, sending...
I20240501 19:56:39.980310 26924 ts_manager.cc:194] Registered new tserver with Master: 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:39.981838 26924 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.1:44325
W20240501 19:56:40.070813 27115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:40.071058 27115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:40.071152 27115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:40.076195 27115 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:40.076292 27115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:40.076449 27115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.2
I20240501 19:56:40.082458 27115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.20.210.2
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.20.210.62:43635
--never_fsync=true
--heap_profile_path=/tmp/kudu.27115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.2
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:40.083138 27115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:40.084192 27115 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:40.092682 27120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:40.093465 27123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:40.094275 27121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:40.094998 27115 server_base.cc:1034] running on GCE node
I20240501 19:56:40.095443 27115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:40.096149 27115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:40.097332 27115 hybrid_clock.cc:648] HybridClock initialized: now 1714593400097315 us; error 34 us; skew 500 ppm
I20240501 19:56:40.099551 27115 webserver.cc:460] Webserver started at http://127.20.210.2:34635/ using document root <none> and password file <none>
I20240501 19:56:40.100178 27115 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:40.100255 27115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:40.100483 27115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:40.102695 27115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.103271 27115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.107519 27115 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20240501 19:56:40.110183 27129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.111368 27115 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20240501 19:56:40.111493 27115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.111618 27115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:40.141949 27115 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:40.142700 27115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:40.142866 27115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:40.143529 27115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:40.144802 27115 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:40.144874 27115 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.144929 27115 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:40.144971 27115 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.157114 27115 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.2:43113
I20240501 19:56:40.157141 27242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.2:43113 every 8 connection(s)
I20240501 19:56:40.158452 27115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 19:56:40.167392 27243 heartbeater.cc:347] Connected to a master server at 127.20.210.62:43635
I20240501 19:56:40.167610 27243 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:40.167747 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 27115
I20240501 19:56:40.167870 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 19:56:40.168208 27243 heartbeater.cc:511] Master 127.20.210.62:43635 requested a full tablet report, sending...
I20240501 19:56:40.169322 26923 ts_manager.cc:194] Registered new tserver with Master: ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:40.169871 26923 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.2:57703
I20240501 19:56:40.170287 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.3:0
--local_ip_for_outbound_sockets=127.20.210.3
--webserver_interface=127.20.210.3
--webserver_port=0
--tserver_master_addrs=127.20.210.62:43635
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240501 19:56:40.270579 27246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:40.270790 27246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:40.270862 27246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:40.273748 27246 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:40.273814 27246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:40.273903 27246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.3
I20240501 19:56:40.277334 27246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.20.210.3
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.20.210.62:43635
--never_fsync=true
--heap_profile_path=/tmp/kudu.27246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.3
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:40.278059 27246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:40.279045 27246 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:40.287067 27252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:40.287824 27251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:40.288554 27254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:40.289330 27246 server_base.cc:1034] running on GCE node
I20240501 19:56:40.289734 27246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:40.290305 27246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:40.291472 27246 hybrid_clock.cc:648] HybridClock initialized: now 1714593400291414 us; error 70 us; skew 500 ppm
I20240501 19:56:40.293265 27246 webserver.cc:460] Webserver started at http://127.20.210.3:39347/ using document root <none> and password file <none>
I20240501 19:56:40.293745 27246 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:40.293798 27246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:40.294003 27246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:40.295560 27246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "3c3f70fdad7b438a841f74fc638d17f5"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.295980 27246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "3c3f70fdad7b438a841f74fc638d17f5"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.298915 27246 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240501 19:56:40.300781 27260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.301643 27246 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240501 19:56:40.301731 27246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "3c3f70fdad7b438a841f74fc638d17f5"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.301816 27246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:40.317409 27246 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:40.317873 27246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:40.318014 27246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:40.318522 27246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:40.319312 27246 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:40.319355 27246 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.319392 27246 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:40.319424 27246 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.326998 27246 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.3:46155
I20240501 19:56:40.327049 27373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.3:46155 every 8 connection(s)
I20240501 19:56:40.327771 27246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 19:56:40.335019 27374 heartbeater.cc:347] Connected to a master server at 127.20.210.62:43635
I20240501 19:56:40.335204 27374 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:40.335661 27374 heartbeater.cc:511] Master 127.20.210.62:43635 requested a full tablet report, sending...
I20240501 19:56:40.335675 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 27246
I20240501 19:56:40.335788 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 19:56:40.336591 26923 ts_manager.cc:194] Registered new tserver with Master: 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:40.337213 26923 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.3:47723
I20240501 19:56:40.337620 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.4:0
--local_ip_for_outbound_sockets=127.20.210.4
--webserver_interface=127.20.210.4
--webserver_port=0
--tserver_master_addrs=127.20.210.62:43635
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240501 19:56:40.434362 27377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:40.434531 27377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:40.434599 27377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:40.437330 27377 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:40.437388 27377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:40.437471 27377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.4
I20240501 19:56:40.440725 27377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.4:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.20.210.4
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.20.210.62:43635
--never_fsync=true
--heap_profile_path=/tmp/kudu.27377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.4
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:40.441433 27377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:40.442402 27377 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:40.450269 27382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:40.451016 27383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:40.451805 27385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:40.452535 27377 server_base.cc:1034] running on GCE node
I20240501 19:56:40.452899 27377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:40.453408 27377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:40.454581 27377 hybrid_clock.cc:648] HybridClock initialized: now 1714593400454565 us; error 34 us; skew 500 ppm
I20240501 19:56:40.456344 27377 webserver.cc:460] Webserver started at http://127.20.210.4:43239/ using document root <none> and password file <none>
I20240501 19:56:40.456848 27377 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:40.456905 27377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:40.457065 27377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:40.458606 27377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "e5057264da53436ea17bad01c66a7fbf"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.459061 27377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "e5057264da53436ea17bad01c66a7fbf"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.462013 27377 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 19:56:40.463974 27391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.464865 27377 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:40.464973 27377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "e5057264da53436ea17bad01c66a7fbf"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.465047 27377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:40.476876 27377 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:40.477331 27377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:40.477468 27377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:40.477890 27377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:40.478663 27377 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:40.478708 27377 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.478757 27377 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:40.478787 27377 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.486436 27377 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.4:37115
I20240501 19:56:40.486474 27504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.4:37115 every 8 connection(s)
I20240501 19:56:40.487196 27377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 19:56:40.492385 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 27377
I20240501 19:56:40.492511 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 19:56:40.494678 27505 heartbeater.cc:347] Connected to a master server at 127.20.210.62:43635
I20240501 19:56:40.494859 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.5:0
--local_ip_for_outbound_sockets=127.20.210.5
--webserver_interface=127.20.210.5
--webserver_port=0
--tserver_master_addrs=127.20.210.62:43635
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 19:56:40.494879 27505 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:40.495332 27505 heartbeater.cc:511] Master 127.20.210.62:43635 requested a full tablet report, sending...
I20240501 19:56:40.496567 26923 ts_manager.cc:194] Registered new tserver with Master: e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:40.497020 26923 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.4:60939
W20240501 19:56:40.590219 27508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:40.590454 27508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:40.590523 27508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:40.593376 27508 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:40.593438 27508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:40.593528 27508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.5
I20240501 19:56:40.597002 27508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.5:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.20.210.5
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.20.210.62:43635
--never_fsync=true
--heap_profile_path=/tmp/kudu.27508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.5
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:40.597723 27508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:40.598685 27508 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:40.606424 27516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:40.607170 27514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:40.607836 27513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:40.608539 27508 server_base.cc:1034] running on GCE node
I20240501 19:56:40.608880 27508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:40.609318 27508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:40.610462 27508 hybrid_clock.cc:648] HybridClock initialized: now 1714593400610446 us; error 28 us; skew 500 ppm
I20240501 19:56:40.612171 27508 webserver.cc:460] Webserver started at http://127.20.210.5:34519/ using document root <none> and password file <none>
I20240501 19:56:40.612654 27508 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:40.612708 27508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:40.612906 27508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:40.614449 27508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "724bae28280242c6982c4f8e315b0d0b"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.614990 27508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "724bae28280242c6982c4f8e315b0d0b"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.617899 27508 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 19:56:40.619904 27522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.620689 27508 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:40.620774 27508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "724bae28280242c6982c4f8e315b0d0b"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.620851 27508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:40.636963 27508 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:40.637399 27508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:40.637531 27508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:40.637957 27508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:40.638670 27508 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:40.638713 27508 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.638769 27508 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:40.638783 27508 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.646795 27508 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.5:42343
I20240501 19:56:40.646837 27635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.5:42343 every 8 connection(s)
I20240501 19:56:40.647624 27508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 19:56:40.650120 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 27508
I20240501 19:56:40.650228 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240501 19:56:40.652284 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.6:0
--local_ip_for_outbound_sockets=127.20.210.6
--webserver_interface=127.20.210.6
--webserver_port=0
--tserver_master_addrs=127.20.210.62:43635
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 19:56:40.656134 27636 heartbeater.cc:347] Connected to a master server at 127.20.210.62:43635
I20240501 19:56:40.656301 27636 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:40.656724 27636 heartbeater.cc:511] Master 127.20.210.62:43635 requested a full tablet report, sending...
I20240501 19:56:40.657558 26923 ts_manager.cc:194] Registered new tserver with Master: 724bae28280242c6982c4f8e315b0d0b (127.20.210.5:42343)
I20240501 19:56:40.658071 26923 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.5:42719
W20240501 19:56:40.748086 27639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:40.748265 27639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:40.748312 27639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:40.751328 27639 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:40.751385 27639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:40.751446 27639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.6
I20240501 19:56:40.754563 27639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.6:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.20.210.6
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.20.210.62:43635
--never_fsync=true
--heap_profile_path=/tmp/kudu.27639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.6
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:40.755308 27639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:40.756276 27639 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:40.763909 27644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:40.764629 27645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:40.765309 27647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:40.766109 27639 server_base.cc:1034] running on GCE node
I20240501 19:56:40.766450 27639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:40.766912 27639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:40.768086 27639 hybrid_clock.cc:648] HybridClock initialized: now 1714593400768070 us; error 31 us; skew 500 ppm
I20240501 19:56:40.769764 27639 webserver.cc:460] Webserver started at http://127.20.210.6:45193/ using document root <none> and password file <none>
I20240501 19:56:40.770273 27639 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:40.770330 27639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:40.770488 27639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:40.771931 27639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "7ccb4fe66ab942d399551b10f5ed32ea"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.772329 27639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "7ccb4fe66ab942d399551b10f5ed32ea"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.775179 27639 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20240501 19:56:40.777016 27653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.777787 27639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240501 19:56:40.777886 27639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "7ccb4fe66ab942d399551b10f5ed32ea"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.777953 27639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:40.792562 27639 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:40.792953 27639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:40.793073 27639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:40.793450 27639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:40.794091 27639 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:40.794132 27639 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.794167 27639 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:40.794194 27639 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.801314 27639 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.6:38529
I20240501 19:56:40.801339 27766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.6:38529 every 8 connection(s)
I20240501 19:56:40.802038 27639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240501 19:56:40.807355 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 27639
I20240501 19:56:40.807457 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240501 19:56:40.809413 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.7:0
--local_ip_for_outbound_sockets=127.20.210.7
--webserver_interface=127.20.210.7
--webserver_port=0
--tserver_master_addrs=127.20.210.62:43635
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 19:56:40.809592 27767 heartbeater.cc:347] Connected to a master server at 127.20.210.62:43635
I20240501 19:56:40.809762 27767 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:40.810159 27767 heartbeater.cc:511] Master 127.20.210.62:43635 requested a full tablet report, sending...
I20240501 19:56:40.810978 26923 ts_manager.cc:194] Registered new tserver with Master: 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529)
I20240501 19:56:40.811573 26923 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.6:44869
W20240501 19:56:40.904022 27770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:40.904194 27770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:40.904242 27770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:40.907160 27770 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:40.907212 27770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:40.907305 27770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.7
I20240501 19:56:40.910436 27770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.7:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.20.210.7
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.20.210.62:43635
--never_fsync=true
--heap_profile_path=/tmp/kudu.27770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.7
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:40.911108 27770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:40.912036 27770 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:40.919703 27775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:40.920404 27776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:40.921041 27778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:40.921726 27770 server_base.cc:1034] running on GCE node
I20240501 19:56:40.922017 27770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:40.922472 27770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:40.923609 27770 hybrid_clock.cc:648] HybridClock initialized: now 1714593400923597 us; error 26 us; skew 500 ppm
I20240501 19:56:40.925280 27770 webserver.cc:460] Webserver started at http://127.20.210.7:36437/ using document root <none> and password file <none>
I20240501 19:56:40.925740 27770 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:40.925791 27770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:40.925940 27770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:40.927367 27770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.927810 27770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.930814 27770 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 19:56:40.932610 27784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.933456 27770 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:40.933537 27770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:40.933599 27770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:40.956679 27770 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:40.957077 27770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:40.957185 27770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:40.957566 27770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:40.958279 27770 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:40.958326 27770 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.958359 27770 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:40.958374 27770 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:40.966153 27770 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.7:33695
I20240501 19:56:40.966176 27897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.7:33695 every 8 connection(s)
I20240501 19:56:40.966964 27770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
W20240501 19:56:40.970482 27107 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.20.210.1:33375
I20240501 19:56:40.974638 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 27770
I20240501 19:56:40.974799 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240501 19:56:40.975247 27898 heartbeater.cc:347] Connected to a master server at 127.20.210.62:43635
I20240501 19:56:40.975443 27898 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:40.975971 27898 heartbeater.cc:511] Master 127.20.210.62:43635 requested a full tablet report, sending...
I20240501 19:56:40.976786 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.8:0
--local_ip_for_outbound_sockets=127.20.210.8
--webserver_interface=127.20.210.8
--webserver_port=0
--tserver_master_addrs=127.20.210.62:43635
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 19:56:40.976804 26923 ts_manager.cc:194] Registered new tserver with Master: ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695)
I20240501 19:56:40.977247 26923 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.7:39755
I20240501 19:56:40.984225 27112 heartbeater.cc:503] Master 127.20.210.62:43635 was elected leader, sending a full tablet report...
W20240501 19:56:41.070063 27901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:41.070277 27901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:41.070333 27901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:41.072850 27901 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:41.072901 27901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:41.072959 27901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.8
I20240501 19:56:41.076121 27901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.8:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.20.210.8
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.20.210.62:43635
--never_fsync=true
--heap_profile_path=/tmp/kudu.27901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.8
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:41.076725 27901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:41.077556 27901 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:41.085220 27907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:41.085917 27906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:41.086639 27909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:41.087356 27901 server_base.cc:1034] running on GCE node
I20240501 19:56:41.087641 27901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:41.088038 27901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:41.089200 27901 hybrid_clock.cc:648] HybridClock initialized: now 1714593401089174 us; error 36 us; skew 500 ppm
I20240501 19:56:41.090901 27901 webserver.cc:460] Webserver started at http://127.20.210.8:41901/ using document root <none> and password file <none>
I20240501 19:56:41.091398 27901 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:41.091450 27901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:41.091593 27901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:41.093037 27901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "474eb4491dcb47f4a2a2f6e149bc39c1"
format_stamp: "Formatted at 2024-05-01 19:56:41 on dist-test-slave-vgh4"
I20240501 19:56:41.093420 27901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "474eb4491dcb47f4a2a2f6e149bc39c1"
format_stamp: "Formatted at 2024-05-01 19:56:41 on dist-test-slave-vgh4"
I20240501 19:56:41.096349 27901 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 19:56:41.098172 27915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:41.099063 27901 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:41.099146 27901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "474eb4491dcb47f4a2a2f6e149bc39c1"
format_stamp: "Formatted at 2024-05-01 19:56:41 on dist-test-slave-vgh4"
I20240501 19:56:41.099222 27901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:41.110880 27901 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:41.111385 27901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:41.111518 27901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:41.111954 27901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:41.112668 27901 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:41.112711 27901 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:41.112749 27901 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:41.112787 27901 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:41.120776 27901 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.8:44887
I20240501 19:56:41.120796 28028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.8:44887 every 8 connection(s)
I20240501 19:56:41.121539 27901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 19:56:41.129148 28029 heartbeater.cc:347] Connected to a master server at 127.20.210.62:43635
I20240501 19:56:41.129312 28029 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:41.129696 28029 heartbeater.cc:511] Master 127.20.210.62:43635 requested a full tablet report, sending...
I20240501 19:56:41.130571 26923 ts_manager.cc:194] Registered new tserver with Master: 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8:44887)
I20240501 19:56:41.131076 26923 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.8:46659
I20240501 19:56:41.131639 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 27901
I20240501 19:56:41.131736 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240501 19:56:41.133714 21320 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240501 19:56:41.142521 21320 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240501 19:56:41.142597 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 27508
I20240501 19:56:41.149300 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 27639
I20240501 19:56:41.158087 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 27770
I20240501 19:56:41.165478 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 27901
I20240501 19:56:41.171550 27243 heartbeater.cc:503] Master 127.20.210.62:43635 was elected leader, sending a full tablet report...
I20240501 19:56:41.339057 27374 heartbeater.cc:503] Master 127.20.210.62:43635 was elected leader, sending a full tablet report...
I20240501 19:56:41.498586 27505 heartbeater.cc:503] Master 127.20.210.62:43635 was elected leader, sending a full tablet report...
I20240501 19:56:44.922116 26923 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50992:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:44.940760 27177 tablet_service.cc:1467] Processing CreateTablet for tablet 3c80c1dcbcd3439ba75066259557e716 (DEFAULT_TABLE table=rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:44.940739 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 199f9529870242f599358995c19250b2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:44.940752 27308 tablet_service.cc:1467] Processing CreateTablet for tablet 692732b291b441029826cf20c9a0ecb1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:44.941278 27046 tablet_service.cc:1467] Processing CreateTablet for tablet 692732b291b441029826cf20c9a0ecb1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:44.941576 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199f9529870242f599358995c19250b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:44.941576 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c80c1dcbcd3439ba75066259557e716. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:44.941576 27308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692732b291b441029826cf20c9a0ecb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:44.942499 27045 tablet_service.cc:1467] Processing CreateTablet for tablet 3c80c1dcbcd3439ba75066259557e716 (DEFAULT_TABLE table=rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:44.942725 27045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c80c1dcbcd3439ba75066259557e716. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:44.942940 27438 tablet_service.cc:1467] Processing CreateTablet for tablet 199f9529870242f599358995c19250b2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:44.943629 27438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199f9529870242f599358995c19250b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:44.943676 27176 tablet_service.cc:1467] Processing CreateTablet for tablet 199f9529870242f599358995c19250b2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:44.943953 27176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199f9529870242f599358995c19250b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:44.946322 27439 tablet_service.cc:1467] Processing CreateTablet for tablet 692732b291b441029826cf20c9a0ecb1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:44.946534 27439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692732b291b441029826cf20c9a0ecb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:44.940752 27307 tablet_service.cc:1467] Processing CreateTablet for tablet 3c80c1dcbcd3439ba75066259557e716 (DEFAULT_TABLE table=rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:44.946772 27046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692732b291b441029826cf20c9a0ecb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:44.946791 27307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c80c1dcbcd3439ba75066259557e716. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:44.950171 28049 tablet_bootstrap.cc:492] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:44.950500 28050 tablet_bootstrap.cc:492] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:44.952061 28049 tablet_bootstrap.cc:654] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:44.952234 28050 tablet_bootstrap.cc:654] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:44.952638 28052 tablet_bootstrap.cc:492] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:44.952857 28050 log.cc:824] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:44.952854 28049 log.cc:824] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:44.953449 28051 tablet_bootstrap.cc:492] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:44.954296 28052 tablet_bootstrap.cc:654] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:44.954366 28050 tablet_bootstrap.cc:492] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:44.954393 28049 tablet_bootstrap.cc:492] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:44.954535 28049 ts_tablet_manager.cc:1367] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240501 19:56:44.954550 28050 ts_tablet_manager.cc:1367] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240501 19:56:44.955103 28052 log.cc:824] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:44.955528 28051 tablet_bootstrap.cc:654] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:44.956234 28051 log.cc:824] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:44.957305 28052 tablet_bootstrap.cc:492] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:44.957435 28052 ts_tablet_manager.cc:1367] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20240501 19:56:44.957656 28051 tablet_bootstrap.cc:492] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:44.957772 28051 ts_tablet_manager.cc:1367] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 19:56:44.957849 28049 raft_consensus.cc:348] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:44.958009 28049 raft_consensus.cc:374] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:44.958051 28049 raft_consensus.cc:729] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:44.957967 28050 raft_consensus.cc:348] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:44.958112 28050 raft_consensus.cc:374] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:44.958151 28050 raft_consensus.cc:729] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:44.958420 28049 consensus_queue.cc:260] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:44.958487 28050 consensus_queue.cc:260] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:44.959190 28050 ts_tablet_manager.cc:1398] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240501 19:56:44.959209 28049 ts_tablet_manager.cc:1398] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240501 19:56:44.960207 28050 tablet_bootstrap.cc:492] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:44.960209 28049 tablet_bootstrap.cc:492] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:44.960312 28051 raft_consensus.cc:348] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:44.960454 28051 raft_consensus.cc:374] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:44.960393 28052 raft_consensus.cc:348] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:44.960496 28051 raft_consensus.cc:729] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:44.960543 28052 raft_consensus.cc:374] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:44.960597 28052 raft_consensus.cc:729] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:44.960806 28051 consensus_queue.cc:260] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:44.960945 28050 tablet_bootstrap.cc:654] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:44.961021 28049 tablet_bootstrap.cc:654] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:44.961490 28051 ts_tablet_manager.cc:1398] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20240501 19:56:44.961753 28051 tablet_bootstrap.cc:492] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:44.961695 28052 consensus_queue.cc:260] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:44.962399 28052 ts_tablet_manager.cc:1398] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20240501 19:56:44.962620 28051 tablet_bootstrap.cc:654] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:44.962675 28052 tablet_bootstrap.cc:492] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:44.963511 28052 tablet_bootstrap.cc:654] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:44.965154 28049 tablet_bootstrap.cc:492] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:44.965154 28050 tablet_bootstrap.cc:492] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:44.965235 28050 ts_tablet_manager.cc:1367] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 19:56:44.965263 28049 ts_tablet_manager.cc:1367] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 19:56:44.965745 28049 raft_consensus.cc:348] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:44.965745 28050 raft_consensus.cc:348] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:44.965848 28050 raft_consensus.cc:374] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:44.965875 28049 raft_consensus.cc:374] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:44.965909 28050 raft_consensus.cc:729] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:44.965932 28049 raft_consensus.cc:729] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:44.966055 28050 consensus_queue.cc:260] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:44.966064 28049 consensus_queue.cc:260] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:44.966444 28049 ts_tablet_manager.cc:1398] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:44.966466 28050 ts_tablet_manager.cc:1398] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:44.967111 28052 tablet_bootstrap.cc:492] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:44.967216 28052 ts_tablet_manager.cc:1367] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 19:56:44.967255 28051 tablet_bootstrap.cc:492] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:44.967331 28051 ts_tablet_manager.cc:1367] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240501 19:56:44.967721 28052 raft_consensus.cc:348] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:44.967834 28052 raft_consensus.cc:374] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:44.967813 28051 raft_consensus.cc:348] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:44.967922 28051 raft_consensus.cc:374] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:44.967960 28051 raft_consensus.cc:729] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:44.967958 28052 raft_consensus.cc:729] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:44.968072 28051 consensus_queue.cc:260] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:44.968080 28052 consensus_queue.cc:260] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:44.968333 28052 ts_tablet_manager.cc:1398] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:44.968346 28051 ts_tablet_manager.cc:1398] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:44.968446 28051 tablet_bootstrap.cc:492] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:44.969115 28051 tablet_bootstrap.cc:654] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:44.971204 28051 tablet_bootstrap.cc:492] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:44.971287 28051 ts_tablet_manager.cc:1367] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:44.971684 28051 raft_consensus.cc:348] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:44.971779 28051 raft_consensus.cc:374] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:44.971809 28051 raft_consensus.cc:729] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:44.971930 28051 consensus_queue.cc:260] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:44.972162 28051 ts_tablet_manager.cc:1398] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240501 19:56:44.973502 27113 tablet.cc:2369] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 19:56:44.990733 27506 tablet.cc:2369] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:56:45.077209 28057 raft_consensus.cc:482] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:45.077378 28057 raft_consensus.cc:504] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:45.078394 28057 leader_election.cc:288] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
W20240501 19:56:45.081832 27375 tablet.cc:2369] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:56:45.082422 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:45.082429 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:45.082662 27066 raft_consensus.cc:2454] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:45.082679 27459 raft_consensus.cc:2454] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:45.083086 27262 leader_election.cc:302] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, 3c3f70fdad7b438a841f74fc638d17f5; no voters:
I20240501 19:56:45.083282 28057 raft_consensus.cc:2781] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:45.083344 28057 raft_consensus.cc:482] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:45.083393 28057 raft_consensus.cc:3037] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.084348 28057 raft_consensus.cc:504] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:45.084662 28057 leader_election.cc:288] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Requested vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:45.084937 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:45.084988 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:45.085026 27459 raft_consensus.cc:3037] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.085134 27066 raft_consensus.cc:3037] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.085788 27459 raft_consensus.cc:2454] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:45.085891 27066 raft_consensus.cc:2454] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:45.086077 27264 leader_election.cc:302] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:45.086277 28057 raft_consensus.cc:2781] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:45.086470 28057 raft_consensus.cc:686] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:45.086802 28057 consensus_queue.cc:237] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:45.089571 26922 catalog_manager.cc:5484] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } }
W20240501 19:56:45.162912 27244 tablet.cc:2369] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:56:45.222345 28057 raft_consensus.cc:482] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:45.222458 28057 raft_consensus.cc:504] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:45.222858 28057 leader_election.cc:288] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:45.223366 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c80c1dcbcd3439ba75066259557e716" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:45.223495 27066 raft_consensus.cc:2454] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:45.223799 27262 leader_election.cc:302] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, 3c3f70fdad7b438a841f74fc638d17f5; no voters:
I20240501 19:56:45.223984 28057 raft_consensus.cc:2781] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:45.224040 28057 raft_consensus.cc:482] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:45.224074 28057 raft_consensus.cc:3037] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.225193 28057 raft_consensus.cc:504] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:45.225618 28057 leader_election.cc:288] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Requested vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:45.225860 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c80c1dcbcd3439ba75066259557e716" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:45.225994 27066 raft_consensus.cc:3037] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.226433 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c80c1dcbcd3439ba75066259557e716" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:45.226462 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c80c1dcbcd3439ba75066259557e716" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:45.226619 27196 raft_consensus.cc:3037] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.227002 27066 raft_consensus.cc:2454] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:45.227229 27262 leader_election.cc:302] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, 3c3f70fdad7b438a841f74fc638d17f5; no voters:
I20240501 19:56:45.227380 28057 raft_consensus.cc:2781] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:45.227474 28057 raft_consensus.cc:686] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:45.227645 28057 consensus_queue.cc:237] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:45.227737 27196 raft_consensus.cc:2454] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:45.229117 26922 catalog_manager.cc:5484] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:45.245936 28059 raft_consensus.cc:482] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:45.246017 28059 raft_consensus.cc:504] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:45.246991 28059 leader_election.cc:288] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:45.247198 28058 raft_consensus.cc:482] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:45.247300 28058 raft_consensus.cc:504] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:45.248250 28058 leader_election.cc:288] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:45.250633 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199f9529870242f599358995c19250b2" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:45.250742 27459 raft_consensus.cc:2454] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 0.
I20240501 19:56:45.250953 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199f9529870242f599358995c19250b2" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:45.251085 27196 raft_consensus.cc:2454] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 0.
I20240501 19:56:45.251195 27002 leader_election.cc:302] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:45.251446 28059 raft_consensus.cc:2781] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:45.251506 28059 raft_consensus.cc:482] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:45.251547 28059 raft_consensus.cc:3037] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.252386 28059 raft_consensus.cc:504] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:45.252681 28059 leader_election.cc:288] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 election: Requested vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:45.252936 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199f9529870242f599358995c19250b2" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:45.252969 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199f9529870242f599358995c19250b2" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:45.252969 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199f9529870242f599358995c19250b2" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:45.253033 27196 raft_consensus.cc:2454] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 0.
I20240501 19:56:45.253057 27459 raft_consensus.cc:3037] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.253863 27459 raft_consensus.cc:2454] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 1.
I20240501 19:56:45.253867 27392 leader_election.cc:302] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5057264da53436ea17bad01c66a7fbf, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:45.253979 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199f9529870242f599358995c19250b2" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:45.254096 28058 raft_consensus.cc:2741] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20240501 19:56:45.254128 27066 raft_consensus.cc:2379] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5057264da53436ea17bad01c66a7fbf in current term 1: Already voted for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in this term.
I20240501 19:56:45.254340 27002 leader_election.cc:302] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, e5057264da53436ea17bad01c66a7fbf; no voters: ef2d33d636574e8fbb9d3b90a15035d6
I20240501 19:56:45.254485 28059 raft_consensus.cc:2781] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:45.254644 28059 raft_consensus.cc:686] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Becoming Leader. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Running, Role: LEADER
I20240501 19:56:45.254917 28059 consensus_queue.cc:237] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:45.257241 26922 catalog_manager.cc:5484] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: term changed from 0 to 1, leader changed from <none> to 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1). New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } }
W20240501 19:56:45.394762 28078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:45.394940 28078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:45.397732 28078 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240501 19:56:45.470852 27459 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:45.471455 27197 raft_consensus.cc:3037] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.472620 27197 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:45.473367 27459 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:45.473481 28059 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:45.473716 28076 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:45.474365 28057 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:45.475224 27066 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:45.475510 27065 raft_consensus.cc:1261] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:45.475664 28057 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:45.477681 28057 consensus_queue.cc:1035] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:45.480396 27197 raft_consensus.cc:1261] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:45.480772 28057 consensus_queue.cc:1035] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:45.489470 28116 mvcc.cc:205] Tried to move back new op lower bound from 7022974588817141760 to 7022974587926351872. Current Snapshot: MvccSnapshot[applied={T|T < 7022974588801085440}]
I20240501 19:56:45.492993 28111 mvcc.cc:205] Tried to move back new op lower bound from 7022974588811730944 to 7022974587238354944. Current Snapshot: MvccSnapshot[applied={T|T < 7022974588811730944}]
I20240501 19:56:45.512702 28120 mvcc.cc:205] Tried to move back new op lower bound from 7022974588830269440 to 7022974587813859328. Current Snapshot: MvccSnapshot[applied={T|T < 7022974588810821632}]
INSERT report
rows total: 2000
time total: 101.552 ms
time per row: 0.0507761 ms
I20240501 19:56:45.538136 26924 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50992:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:45.542987 27308 tablet_service.cc:1467] Processing CreateTablet for tablet 0f417f4001bc48158187ed262f7737ab (DEFAULT_TABLE table=rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:45.543015 27438 tablet_service.cc:1467] Processing CreateTablet for tablet 0f417f4001bc48158187ed262f7737ab (DEFAULT_TABLE table=rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:45.543259 27308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f417f4001bc48158187ed262f7737ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:45.543279 27438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f417f4001bc48158187ed262f7737ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:45.543985 27439 tablet_service.cc:1467] Processing CreateTablet for tablet e3b87a633efe4400931f1e0a566c8f14 (DEFAULT_TABLE table=rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:45.544191 27439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b87a633efe4400931f1e0a566c8f14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:45.545051 28136 tablet_bootstrap.cc:492] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:45.545339 28137 tablet_bootstrap.cc:492] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:45.545902 28136 tablet_bootstrap.cc:654] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:45.545933 27177 tablet_service.cc:1467] Processing CreateTablet for tablet 0f417f4001bc48158187ed262f7737ab (DEFAULT_TABLE table=rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:45.546186 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f417f4001bc48158187ed262f7737ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:45.546398 28137 tablet_bootstrap.cc:654] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:45.546533 27176 tablet_service.cc:1467] Processing CreateTablet for tablet e3b87a633efe4400931f1e0a566c8f14 (DEFAULT_TABLE table=rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:45.546783 27046 tablet_service.cc:1467] Processing CreateTablet for tablet c9f961f0554c45a1a3f31d5a8b31a46d (DEFAULT_TABLE table=rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:45.546908 27176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b87a633efe4400931f1e0a566c8f14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:45.547039 27046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9f961f0554c45a1a3f31d5a8b31a46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:45.548599 28140 tablet_bootstrap.cc:492] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:45.548769 27175 tablet_service.cc:1467] Processing CreateTablet for tablet c9f961f0554c45a1a3f31d5a8b31a46d (DEFAULT_TABLE table=rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:45.548983 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9f961f0554c45a1a3f31d5a8b31a46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:45.549120 28141 tablet_bootstrap.cc:492] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:45.549331 28140 tablet_bootstrap.cc:654] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:45.549875 28141 tablet_bootstrap.cc:654] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:45.550077 28137 tablet_bootstrap.cc:492] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:45.550165 28137 ts_tablet_manager.cc:1367] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240501 19:56:45.550597 28137 raft_consensus.cc:348] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:45.550714 28137 raft_consensus.cc:374] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:45.550755 28137 raft_consensus.cc:729] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:45.550881 28141 tablet_bootstrap.cc:492] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:45.550868 28137 consensus_queue.cc:260] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:45.550956 28141 ts_tablet_manager.cc:1367] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:45.551218 28137 ts_tablet_manager.cc:1398] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:45.551388 28137 tablet_bootstrap.cc:492] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:45.551502 28141 raft_consensus.cc:348] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:45.551584 28141 raft_consensus.cc:374] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:45.551609 28141 raft_consensus.cc:729] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:45.551586 27308 tablet_service.cc:1467] Processing CreateTablet for tablet e3b87a633efe4400931f1e0a566c8f14 (DEFAULT_TABLE table=rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:45.551663 27307 tablet_service.cc:1467] Processing CreateTablet for tablet c9f961f0554c45a1a3f31d5a8b31a46d (DEFAULT_TABLE table=rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:45.551731 28141 consensus_queue.cc:260] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:45.551785 27308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b87a633efe4400931f1e0a566c8f14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:45.552129 28137 tablet_bootstrap.cc:654] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:45.553109 28140 tablet_bootstrap.cc:492] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:45.553189 28140 ts_tablet_manager.cc:1367] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 19:56:45.553244 28141 ts_tablet_manager.cc:1398] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:45.553567 28140 raft_consensus.cc:348] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:45.553668 28140 raft_consensus.cc:374] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:45.553699 28140 raft_consensus.cc:729] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:45.553838 28140 consensus_queue.cc:260] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:45.554158 28140 ts_tablet_manager.cc:1398] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:45.554454 27307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9f961f0554c45a1a3f31d5a8b31a46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:45.554622 28136 tablet_bootstrap.cc:492] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:45.554699 28136 ts_tablet_manager.cc:1367] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240501 19:56:45.555029 28136 raft_consensus.cc:348] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:45.555112 28136 raft_consensus.cc:374] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:45.555164 28136 raft_consensus.cc:729] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:45.555292 28136 consensus_queue.cc:260] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:45.555652 28136 ts_tablet_manager.cc:1398] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:45.555805 28136 tablet_bootstrap.cc:492] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:45.556780 28136 tablet_bootstrap.cc:654] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:45.555447 28141 tablet_bootstrap.cc:492] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:45.557659 28136 tablet_bootstrap.cc:492] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:45.557746 28136 ts_tablet_manager.cc:1367] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:45.557898 28141 tablet_bootstrap.cc:654] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:45.558154 28136 raft_consensus.cc:348] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:45.558264 28136 raft_consensus.cc:374] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:45.558298 28136 raft_consensus.cc:729] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:45.558398 28136 consensus_queue.cc:260] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:45.558614 28136 ts_tablet_manager.cc:1398] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:45.558777 28136 tablet_bootstrap.cc:492] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:45.559330 28141 tablet_bootstrap.cc:492] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:45.559404 28141 ts_tablet_manager.cc:1367] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 19:56:45.559587 28136 tablet_bootstrap.cc:654] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:45.559751 28141 raft_consensus.cc:348] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:45.559826 28141 raft_consensus.cc:374] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:45.559862 28141 raft_consensus.cc:729] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:45.559991 28141 consensus_queue.cc:260] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:45.560225 28141 ts_tablet_manager.cc:1398] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:45.560382 28141 tablet_bootstrap.cc:492] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:45.560964 28136 tablet_bootstrap.cc:492] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:45.561028 28141 tablet_bootstrap.cc:654] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:45.561043 28136 ts_tablet_manager.cc:1367] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:56:45.561398 28136 raft_consensus.cc:348] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:45.561476 28136 raft_consensus.cc:374] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:45.561503 28136 raft_consensus.cc:729] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:45.561674 28137 tablet_bootstrap.cc:492] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:45.561635 28136 consensus_queue.cc:260] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:45.561738 28137 ts_tablet_manager.cc:1367] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20240501 19:56:45.561901 28136 ts_tablet_manager.cc:1398] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:45.562152 28137 raft_consensus.cc:348] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:45.562263 28137 raft_consensus.cc:374] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:45.562301 28137 raft_consensus.cc:729] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:45.562394 28137 consensus_queue.cc:260] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:45.562652 28137 ts_tablet_manager.cc:1398] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:45.562951 28141 tablet_bootstrap.cc:492] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:45.563021 28141 ts_tablet_manager.cc:1367] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:45.563345 28141 raft_consensus.cc:348] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:45.563428 28141 raft_consensus.cc:374] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:45.563464 28141 raft_consensus.cc:729] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:45.563606 28141 consensus_queue.cc:260] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:45.563849 28141 ts_tablet_manager.cc:1398] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:45.576624 28068 raft_consensus.cc:482] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:45.576695 28068 raft_consensus.cc:504] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:45.577041 28068 leader_election.cc:288] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:45.577312 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b87a633efe4400931f1e0a566c8f14" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:45.577338 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b87a633efe4400931f1e0a566c8f14" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:45.577412 27197 raft_consensus.cc:2454] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:45.577432 27459 raft_consensus.cc:2454] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:45.577641 27261 leader_election.cc:302] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:45.577781 28068 raft_consensus.cc:2781] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:45.577833 28068 raft_consensus.cc:482] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:45.577859 28068 raft_consensus.cc:3037] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.578596 28068 raft_consensus.cc:504] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:45.578898 28068 leader_election.cc:288] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Requested vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:45.579156 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b87a633efe4400931f1e0a566c8f14" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:45.579191 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b87a633efe4400931f1e0a566c8f14" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:45.579246 27197 raft_consensus.cc:3037] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.579296 27459 raft_consensus.cc:3037] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.579880 27197 raft_consensus.cc:2454] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:45.579908 27459 raft_consensus.cc:2454] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:45.580103 27261 leader_election.cc:302] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:45.580269 28068 raft_consensus.cc:2781] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:45.580345 28068 raft_consensus.cc:686] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:45.580453 28068 consensus_queue.cc:237] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:45.581610 26922 catalog_manager.cc:5484] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:45.647545 28110 raft_consensus.cc:482] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:45.647661 28110 raft_consensus.cc:504] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:45.648695 28110 leader_election.cc:288] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:45.651906 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f417f4001bc48158187ed262f7737ab" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:45.651961 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f417f4001bc48158187ed262f7737ab" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5" is_pre_election: true
I20240501 19:56:45.652030 27459 raft_consensus.cc:2454] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:45.652151 27328 raft_consensus.cc:2454] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:45.652356 27133 leader_election.cc:302] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5057264da53436ea17bad01c66a7fbf, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:45.652575 28110 raft_consensus.cc:2781] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:45.652627 28110 raft_consensus.cc:482] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:45.652652 28110 raft_consensus.cc:3037] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.653426 28110 raft_consensus.cc:504] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:45.653720 28110 leader_election.cc:288] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:45.653980 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f417f4001bc48158187ed262f7737ab" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:45.654014 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f417f4001bc48158187ed262f7737ab" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:45.654069 27328 raft_consensus.cc:3037] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.654100 27459 raft_consensus.cc:3037] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.654763 27459 raft_consensus.cc:2454] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 1.
I20240501 19:56:45.654769 27328 raft_consensus.cc:2454] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 1.
I20240501 19:56:45.655004 27133 leader_election.cc:302] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5057264da53436ea17bad01c66a7fbf, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:45.655198 28110 raft_consensus.cc:2781] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:45.655386 28110 raft_consensus.cc:686] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Becoming Leader. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Running, Role: LEADER
I20240501 19:56:45.655632 28110 consensus_queue.cc:237] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:45.657410 26922 catalog_manager.cc:5484] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2). New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } }
I20240501 19:56:45.939555 28059 raft_consensus.cc:482] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:45.939677 28059 raft_consensus.cc:504] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:45.940162 28059 leader_election.cc:288] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:45.940546 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9f961f0554c45a1a3f31d5a8b31a46d" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:45.940707 27197 raft_consensus.cc:2454] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 0.
I20240501 19:56:45.940984 26999 leader_election.cc:302] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:45.941155 28059 raft_consensus.cc:2781] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:45.941202 28059 raft_consensus.cc:482] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:45.941228 28059 raft_consensus.cc:3037] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.942126 28059 raft_consensus.cc:504] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:45.942456 28059 leader_election.cc:288] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 election: Requested vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:45.942755 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9f961f0554c45a1a3f31d5a8b31a46d" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:45.942868 27197 raft_consensus.cc:3037] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:45.943574 27197 raft_consensus.cc:2454] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 1.
I20240501 19:56:45.943614 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9f961f0554c45a1a3f31d5a8b31a46d" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5" is_pre_election: true
I20240501 19:56:45.943725 27327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9f961f0554c45a1a3f31d5a8b31a46d" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:45.943768 27328 raft_consensus.cc:2454] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 0.
I20240501 19:56:45.943892 26999 leader_election.cc:302] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:45.944022 28059 raft_consensus.cc:2781] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:45.944098 28059 raft_consensus.cc:686] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Becoming Leader. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Running, Role: LEADER
I20240501 19:56:45.944236 28059 consensus_queue.cc:237] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:45.945818 26922 catalog_manager.cc:5484] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: term changed from 0 to 1, leader changed from <none> to 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1). New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:46.013314 28068 consensus_queue.cc:1035] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:46.016216 28057 consensus_queue.cc:1035] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240501 19:56:46.096326 28156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:46.096529 28156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:46.099206 28156 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240501 19:56:46.132953 28110 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:46.138576 28152 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:46.213649 27327 raft_consensus.cc:3037] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.214041 27196 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:46.214462 28059 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:46.214638 27327 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:46.215191 28059 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 114.723 ms
time per row: 0.0573613 ms
I20240501 19:56:46.251087 26924 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50992:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:46.255576 27046 tablet_service.cc:1467] Processing CreateTablet for tablet 6c498eacc5e947bfb01dfb90c7039df0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.255717 27439 tablet_service.cc:1467] Processing CreateTablet for tablet 6c498eacc5e947bfb01dfb90c7039df0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.255843 27046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c498eacc5e947bfb01dfb90c7039df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.255993 27439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c498eacc5e947bfb01dfb90c7039df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.256286 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 5ecba34e358c41d9b812113c955b6b29 (DEFAULT_TABLE table=rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.256512 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecba34e358c41d9b812113c955b6b29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.256774 27307 tablet_service.cc:1467] Processing CreateTablet for tablet 5ecba34e358c41d9b812113c955b6b29 (DEFAULT_TABLE table=rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.256992 27307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecba34e358c41d9b812113c955b6b29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.258107 28218 tablet_bootstrap.cc:492] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:46.258634 27175 tablet_service.cc:1467] Processing CreateTablet for tablet 6c498eacc5e947bfb01dfb90c7039df0 (DEFAULT_TABLE table=rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.258668 27177 tablet_service.cc:1467] Processing CreateTablet for tablet 37288cafed764ba18fcc07b6e3c7a722 (DEFAULT_TABLE table=rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.258878 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c498eacc5e947bfb01dfb90c7039df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.259018 28218 tablet_bootstrap.cc:654] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.259079 27439 tablet_service.cc:1467] Processing CreateTablet for tablet 5ecba34e358c41d9b812113c955b6b29 (DEFAULT_TABLE table=rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.259089 27438 tablet_service.cc:1467] Processing CreateTablet for tablet 37288cafed764ba18fcc07b6e3c7a722 (DEFAULT_TABLE table=rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.259295 27439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecba34e358c41d9b812113c955b6b29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.259510 27438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37288cafed764ba18fcc07b6e3c7a722. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.260561 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37288cafed764ba18fcc07b6e3c7a722. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.260865 28217 tablet_bootstrap.cc:492] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:46.260990 28221 tablet_bootstrap.cc:492] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:46.261637 28217 tablet_bootstrap.cc:654] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.261860 28221 tablet_bootstrap.cc:654] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.262002 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 37288cafed764ba18fcc07b6e3c7a722 (DEFAULT_TABLE table=rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.262202 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37288cafed764ba18fcc07b6e3c7a722. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.262506 28217 tablet_bootstrap.cc:492] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:46.262583 28217 ts_tablet_manager.cc:1367] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:46.262931 28217 raft_consensus.cc:348] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.263029 28217 raft_consensus.cc:374] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.263062 28217 raft_consensus.cc:729] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.263168 28217 consensus_queue.cc:260] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.263217 28221 tablet_bootstrap.cc:492] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:46.263283 28221 ts_tablet_manager.cc:1367] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:46.263407 28217 ts_tablet_manager.cc:1398] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:46.263553 28217 tablet_bootstrap.cc:492] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:46.263640 28221 raft_consensus.cc:348] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.263744 28221 raft_consensus.cc:374] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.263788 28221 raft_consensus.cc:729] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.264024 28221 consensus_queue.cc:260] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.264056 28219 tablet_bootstrap.cc:492] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:46.264266 28221 ts_tablet_manager.cc:1398] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:46.264334 28217 tablet_bootstrap.cc:654] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.264603 28218 tablet_bootstrap.cc:492] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:46.264678 28218 ts_tablet_manager.cc:1367] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240501 19:56:46.265043 28219 tablet_bootstrap.cc:654] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.265110 28218 raft_consensus.cc:348] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.265204 28218 raft_consensus.cc:374] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.265252 28218 raft_consensus.cc:729] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.265388 28218 consensus_queue.cc:260] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.265636 28217 tablet_bootstrap.cc:492] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:46.265652 28218 ts_tablet_manager.cc:1398] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:46.265691 28217 ts_tablet_manager.cc:1367] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 19:56:46.265832 28218 tablet_bootstrap.cc:492] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:46.266053 28217 raft_consensus.cc:348] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.266144 28217 raft_consensus.cc:374] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.266187 28217 raft_consensus.cc:729] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.266860 28217 consensus_queue.cc:260] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.267060 28218 tablet_bootstrap.cc:654] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.267199 28217 ts_tablet_manager.cc:1398] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:46.267341 28217 tablet_bootstrap.cc:492] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:46.267627 28221 tablet_bootstrap.cc:492] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:46.268168 28218 tablet_bootstrap.cc:492] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:46.268239 28218 ts_tablet_manager.cc:1367] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:46.268455 28217 tablet_bootstrap.cc:654] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.268652 28218 raft_consensus.cc:348] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.268750 28218 raft_consensus.cc:374] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.268790 28218 raft_consensus.cc:729] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.268941 28218 consensus_queue.cc:260] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.269212 28218 ts_tablet_manager.cc:1398] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:46.269351 28218 tablet_bootstrap.cc:492] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:46.269392 28217 tablet_bootstrap.cc:492] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:46.269457 28217 ts_tablet_manager.cc:1367] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:46.269794 28217 raft_consensus.cc:348] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:46.269899 28217 raft_consensus.cc:374] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.269927 28217 raft_consensus.cc:729] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.270016 28217 consensus_queue.cc:260] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:46.270087 28218 tablet_bootstrap.cc:654] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.270262 28217 ts_tablet_manager.cc:1398] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:46.271215 28221 tablet_bootstrap.cc:654] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.271382 28219 tablet_bootstrap.cc:492] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:46.271461 28219 ts_tablet_manager.cc:1367] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240501 19:56:46.271859 28219 raft_consensus.cc:348] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.271962 28219 raft_consensus.cc:374] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.272002 28219 raft_consensus.cc:729] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.272017 28218 tablet_bootstrap.cc:492] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:46.272117 28218 ts_tablet_manager.cc:1367] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240501 19:56:46.272274 28219 consensus_queue.cc:260] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.272469 28218 raft_consensus.cc:348] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:46.272554 28218 raft_consensus.cc:374] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.272605 28218 raft_consensus.cc:729] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.272712 28219 ts_tablet_manager.cc:1398] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:46.272737 28218 consensus_queue.cc:260] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:46.273037 28218 ts_tablet_manager.cc:1398] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:46.273489 28221 tablet_bootstrap.cc:492] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:46.273587 28221 ts_tablet_manager.cc:1367] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 19:56:46.273986 28221 raft_consensus.cc:348] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:46.274085 28221 raft_consensus.cc:374] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.274119 28221 raft_consensus.cc:729] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.274224 28221 consensus_queue.cc:260] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:46.274518 28221 ts_tablet_manager.cc:1398] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:46.288017 28059 raft_consensus.cc:482] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:46.288084 28059 raft_consensus.cc:504] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.288327 28059 leader_election.cc:288] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:46.288625 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c498eacc5e947bfb01dfb90c7039df0" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:46.288631 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c498eacc5e947bfb01dfb90c7039df0" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:46.288719 27196 raft_consensus.cc:2454] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 0.
I20240501 19:56:46.288731 27459 raft_consensus.cc:2454] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 0.
I20240501 19:56:46.288969 27002 leader_election.cc:302] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:46.289108 28059 raft_consensus.cc:2781] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:46.289168 28059 raft_consensus.cc:482] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:46.289206 28059 raft_consensus.cc:3037] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.289824 28059 raft_consensus.cc:504] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.290064 28059 leader_election.cc:288] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 election: Requested vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:46.290356 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c498eacc5e947bfb01dfb90c7039df0" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:46.290362 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c498eacc5e947bfb01dfb90c7039df0" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:46.290455 27196 raft_consensus.cc:3037] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.290462 27459 raft_consensus.cc:3037] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.291236 27196 raft_consensus.cc:2454] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 1.
I20240501 19:56:46.291236 27459 raft_consensus.cc:2454] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 1.
I20240501 19:56:46.291487 26999 leader_election.cc:302] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:46.291649 28059 raft_consensus.cc:2781] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:46.291725 28059 raft_consensus.cc:686] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Becoming Leader. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Running, Role: LEADER
I20240501 19:56:46.291846 28059 consensus_queue.cc:237] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.293042 26921 catalog_manager.cc:5484] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: term changed from 0 to 1, leader changed from <none> to 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1). New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:46.381855 28059 raft_consensus.cc:482] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:46.381947 28059 raft_consensus.cc:504] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:46.382282 28059 leader_election.cc:288] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:46.382609 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37288cafed764ba18fcc07b6e3c7a722" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:46.382627 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37288cafed764ba18fcc07b6e3c7a722" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:46.382738 27459 raft_consensus.cc:2454] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 0.
I20240501 19:56:46.382745 27196 raft_consensus.cc:2454] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 0.
I20240501 19:56:46.382992 27002 leader_election.cc:302] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:46.383153 28059 raft_consensus.cc:2781] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:46.383222 28059 raft_consensus.cc:482] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:46.383267 28059 raft_consensus.cc:3037] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.384033 28059 raft_consensus.cc:504] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:46.384351 28059 leader_election.cc:288] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 election: Requested vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:46.384608 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37288cafed764ba18fcc07b6e3c7a722" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:46.384647 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37288cafed764ba18fcc07b6e3c7a722" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:46.384728 27196 raft_consensus.cc:3037] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.384734 27459 raft_consensus.cc:3037] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.385430 27459 raft_consensus.cc:2454] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 1.
I20240501 19:56:46.385458 27196 raft_consensus.cc:2454] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 1.
I20240501 19:56:46.385717 27002 leader_election.cc:302] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:46.385892 28059 raft_consensus.cc:2781] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:46.385978 28059 raft_consensus.cc:686] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Becoming Leader. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Running, Role: LEADER
I20240501 19:56:46.386109 28059 consensus_queue.cc:237] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:46.387569 26921 catalog_manager.cc:5484] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: term changed from 0 to 1, leader changed from <none> to 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1). New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:46.390031 28057 raft_consensus.cc:482] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:46.390115 28057 raft_consensus.cc:504] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.390457 28057 leader_election.cc:288] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:46.390699 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecba34e358c41d9b812113c955b6b29" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:46.390805 27459 raft_consensus.cc:2454] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:46.390801 27065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecba34e358c41d9b812113c955b6b29" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:46.390906 27065 raft_consensus.cc:2454] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:46.391012 27264 leader_election.cc:302] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:46.391155 28057 raft_consensus.cc:2781] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:46.391203 28057 raft_consensus.cc:482] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:46.391230 28057 raft_consensus.cc:3037] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.391842 28057 raft_consensus.cc:504] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.392127 28057 leader_election.cc:288] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Requested vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:46.392359 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecba34e358c41d9b812113c955b6b29" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:46.392395 27065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecba34e358c41d9b812113c955b6b29" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:46.392437 27459 raft_consensus.cc:3037] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.392479 27065 raft_consensus.cc:3037] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.393019 27459 raft_consensus.cc:2454] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:46.393257 27264 leader_election.cc:302] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:46.393409 28057 raft_consensus.cc:2781] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:46.393489 28057 raft_consensus.cc:686] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:46.393566 27065 raft_consensus.cc:2454] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:46.393597 28057 consensus_queue.cc:237] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.395011 26921 catalog_manager.cc:5484] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } }
W20240501 19:56:46.506340 28233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:46.506600 28233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:46.509202 28233 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
W20240501 19:56:46.567234 27111 acceptor_pool.cc:285] unable to collect diagnostics on RPC socket 127.20.210.1:33375: Invalid argument: socket address is ambiguous
I20240501 19:56:46.575124 27196 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:46.575556 28059 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:46.577359 27459 raft_consensus.cc:1261] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:46.577654 27458 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:46.577847 27065 raft_consensus.cc:1261] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:46.577972 28059 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:46.578295 28195 consensus_queue.cc:1035] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:46.578437 28057 consensus_queue.cc:1035] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:46.581609 27197 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:46.582087 28216 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:46.583657 27457 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:46.589637 28059 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:46.601666 28158 mvcc.cc:205] Tried to move back new op lower bound from 7022974593401544704 to 7022974592589422592. Current Snapshot: MvccSnapshot[applied={T|T < 7022974593401544704 or (T in {7022974593401544704})}]
INSERT report
rows total: 2000
time total: 88.0851 ms
time per row: 0.0440425 ms
I20240501 19:56:46.635571 26918 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50992:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:46.640420 27177 tablet_service.cc:1467] Processing CreateTablet for tablet 8c157f7f72dc4211b256eee0015eed23 (DEFAULT_TABLE table=rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.640659 27307 tablet_service.cc:1467] Processing CreateTablet for tablet 8c157f7f72dc4211b256eee0015eed23 (DEFAULT_TABLE table=rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.640739 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c157f7f72dc4211b256eee0015eed23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.640923 27307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c157f7f72dc4211b256eee0015eed23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.641148 27044 tablet_service.cc:1467] Processing CreateTablet for tablet 8c157f7f72dc4211b256eee0015eed23 (DEFAULT_TABLE table=rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.641382 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c157f7f72dc4211b256eee0015eed23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.642195 28221 tablet_bootstrap.cc:492] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:46.642624 27438 tablet_service.cc:1467] Processing CreateTablet for tablet f149fd8790a0412e81ac49d0ce359ede (DEFAULT_TABLE table=rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.642859 27438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f149fd8790a0412e81ac49d0ce359ede. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.643067 28221 tablet_bootstrap.cc:654] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.643090 27439 tablet_service.cc:1467] Processing CreateTablet for tablet 6ee4db743833421e8120edbec4ba0836 (DEFAULT_TABLE table=rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.643290 27439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee4db743833421e8120edbec4ba0836. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.643352 28219 tablet_bootstrap.cc:492] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:46.644138 28219 tablet_bootstrap.cc:654] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.644505 28218 tablet_bootstrap.cc:492] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:46.645177 28217 tablet_bootstrap.cc:492] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:46.645359 28218 tablet_bootstrap.cc:654] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.645568 28219 tablet_bootstrap.cc:492] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:46.645658 28219 ts_tablet_manager.cc:1367] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:46.646054 28217 tablet_bootstrap.cc:654] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.646144 28219 raft_consensus.cc:348] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:46.646298 28219 raft_consensus.cc:374] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.646333 28219 raft_consensus.cc:729] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.646340 28218 tablet_bootstrap.cc:492] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:46.646422 28218 ts_tablet_manager.cc:1367] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:56:46.646466 28219 consensus_queue.cc:260] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:46.646762 28219 ts_tablet_manager.cc:1398] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:46.646843 28218 raft_consensus.cc:348] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.646951 28218 raft_consensus.cc:374] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.646986 28218 raft_consensus.cc:729] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.647117 28218 consensus_queue.cc:260] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.647337 28218 ts_tablet_manager.cc:1398] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:46.647329 27307 tablet_service.cc:1467] Processing CreateTablet for tablet f149fd8790a0412e81ac49d0ce359ede (DEFAULT_TABLE table=rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.647329 27308 tablet_service.cc:1467] Processing CreateTablet for tablet 6ee4db743833421e8120edbec4ba0836 (DEFAULT_TABLE table=rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.647478 28218 tablet_bootstrap.cc:492] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:46.647544 27307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f149fd8790a0412e81ac49d0ce359ede. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.647948 27308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee4db743833421e8120edbec4ba0836. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.648202 28218 tablet_bootstrap.cc:654] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.648260 27177 tablet_service.cc:1467] Processing CreateTablet for tablet 6ee4db743833421e8120edbec4ba0836 (DEFAULT_TABLE table=rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.648480 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee4db743833421e8120edbec4ba0836. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.649137 28218 tablet_bootstrap.cc:492] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:46.649196 28218 ts_tablet_manager.cc:1367] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:46.649610 28218 raft_consensus.cc:348] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.649696 28218 raft_consensus.cc:374] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.649741 28218 raft_consensus.cc:729] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.649849 28218 consensus_queue.cc:260] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.650043 28218 ts_tablet_manager.cc:1398] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:46.650888 28217 tablet_bootstrap.cc:492] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:46.651022 28217 ts_tablet_manager.cc:1367] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240501 19:56:46.651433 28217 raft_consensus.cc:348] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:46.651523 28217 raft_consensus.cc:374] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.651559 28217 raft_consensus.cc:729] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.651669 28217 consensus_queue.cc:260] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:46.651829 28221 tablet_bootstrap.cc:492] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:46.651916 28221 ts_tablet_manager.cc:1367] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20240501 19:56:46.651916 28217 ts_tablet_manager.cc:1398] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:46.650872 27044 tablet_service.cc:1467] Processing CreateTablet for tablet f149fd8790a0412e81ac49d0ce359ede (DEFAULT_TABLE table=rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:46.652206 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f149fd8790a0412e81ac49d0ce359ede. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:46.652348 28221 raft_consensus.cc:348] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:46.652469 28221 raft_consensus.cc:374] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.652508 28221 raft_consensus.cc:729] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.652623 28221 consensus_queue.cc:260] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:46.652887 28221 ts_tablet_manager.cc:1398] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:46.653012 28221 tablet_bootstrap.cc:492] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:46.653123 28219 tablet_bootstrap.cc:492] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:46.653831 28221 tablet_bootstrap.cc:654] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.655937 28219 tablet_bootstrap.cc:654] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.656654 28217 tablet_bootstrap.cc:492] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:46.656863 28219 tablet_bootstrap.cc:492] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:46.656950 28219 ts_tablet_manager.cc:1367] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 19:56:46.657063 28221 tablet_bootstrap.cc:492] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:46.657136 28221 ts_tablet_manager.cc:1367] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:56:46.657425 28221 raft_consensus.cc:348] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.657500 28221 raft_consensus.cc:374] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.657542 28221 raft_consensus.cc:729] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.657615 28217 tablet_bootstrap.cc:654] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.657361 28219 raft_consensus.cc:348] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.657678 28219 raft_consensus.cc:374] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.657718 28219 raft_consensus.cc:729] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.657840 28219 consensus_queue.cc:260] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.657826 28221 consensus_queue.cc:260] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.658090 28219 ts_tablet_manager.cc:1398] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:46.658210 28219 tablet_bootstrap.cc:492] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:46.658488 28221 ts_tablet_manager.cc:1398] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:46.659147 28217 tablet_bootstrap.cc:492] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:46.659174 28219 tablet_bootstrap.cc:654] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:46.659235 28217 ts_tablet_manager.cc:1367] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240501 19:56:46.659654 28217 raft_consensus.cc:348] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.659754 28217 raft_consensus.cc:374] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.659785 28217 raft_consensus.cc:729] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.659893 28217 consensus_queue.cc:260] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.660105 28217 ts_tablet_manager.cc:1398] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:46.660115 28219 tablet_bootstrap.cc:492] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:46.660177 28219 ts_tablet_manager.cc:1367] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:46.660552 28219 raft_consensus.cc:348] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.660661 28219 raft_consensus.cc:374] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:46.660701 28219 raft_consensus.cc:729] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:46.660818 28219 consensus_queue.cc:260] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.661064 28219 ts_tablet_manager.cc:1398] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:46.697268 28185 raft_consensus.cc:482] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:46.697348 28185 raft_consensus.cc:504] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.697702 28185 leader_election.cc:288] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:46.698076 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee4db743833421e8120edbec4ba0836" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:46.698182 27196 raft_consensus.cc:2454] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 0.
I20240501 19:56:46.698412 27392 leader_election.cc:302] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5057264da53436ea17bad01c66a7fbf, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:46.698616 28185 raft_consensus.cc:2781] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:46.698679 28185 raft_consensus.cc:482] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:46.698734 28185 raft_consensus.cc:3037] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.699524 28185 raft_consensus.cc:504] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.699831 28185 leader_election.cc:288] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 election: Requested vote from peers 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:46.700085 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee4db743833421e8120edbec4ba0836" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:46.700189 27196 raft_consensus.cc:3037] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.701045 27196 raft_consensus.cc:2454] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 1.
I20240501 19:56:46.701292 27392 leader_election.cc:302] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5057264da53436ea17bad01c66a7fbf, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:46.701326 27327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee4db743833421e8120edbec4ba0836" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5" is_pre_election: true
I20240501 19:56:46.701422 27327 raft_consensus.cc:2454] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 0.
I20240501 19:56:46.701453 28185 raft_consensus.cc:2781] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:46.701604 28185 raft_consensus.cc:686] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Becoming Leader. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Running, Role: LEADER
I20240501 19:56:46.701653 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ee4db743833421e8120edbec4ba0836" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:46.701731 27328 raft_consensus.cc:3037] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.701901 28185 consensus_queue.cc:237] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:46.702559 27328 raft_consensus.cc:2454] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 1.
I20240501 19:56:46.704155 26918 catalog_manager.cc:5484] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf reported cstate change: term changed from 0 to 1, leader changed from <none> to e5057264da53436ea17bad01c66a7fbf (127.20.210.4). New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:46.708782 28216 raft_consensus.cc:482] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:46.708873 28216 raft_consensus.cc:504] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:46.709290 28216 leader_election.cc:288] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:46.709539 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c157f7f72dc4211b256eee0015eed23" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:46.709636 27196 raft_consensus.cc:2454] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 0.
I20240501 19:56:46.709692 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c157f7f72dc4211b256eee0015eed23" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5" is_pre_election: true
I20240501 19:56:46.709806 27328 raft_consensus.cc:2454] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 0.
I20240501 19:56:46.709898 26999 leader_election.cc:302] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:46.710063 28076 raft_consensus.cc:2781] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:46.710104 28076 raft_consensus.cc:482] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:46.710140 28076 raft_consensus.cc:3037] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.710821 28076 raft_consensus.cc:504] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:46.711081 28076 leader_election.cc:288] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 election: Requested vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:46.711347 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c157f7f72dc4211b256eee0015eed23" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:46.711351 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c157f7f72dc4211b256eee0015eed23" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:46.711438 27196 raft_consensus.cc:3037] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.711438 27328 raft_consensus.cc:3037] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.712132 27328 raft_consensus.cc:2454] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 1.
I20240501 19:56:46.712132 27196 raft_consensus.cc:2454] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 1.
I20240501 19:56:46.712450 26999 leader_election.cc:302] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:46.712633 28076 raft_consensus.cc:2781] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:46.712720 28076 raft_consensus.cc:686] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Becoming Leader. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Running, Role: LEADER
I20240501 19:56:46.712848 28076 consensus_queue.cc:237] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:46.714215 26918 catalog_manager.cc:5484] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: term changed from 0 to 1, leader changed from <none> to 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1). New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:46.883656 28068 raft_consensus.cc:482] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:46.883767 28068 raft_consensus.cc:504] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.884167 28068 leader_election.cc:288] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:46.884533 27065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f149fd8790a0412e81ac49d0ce359ede" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:46.884541 27458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f149fd8790a0412e81ac49d0ce359ede" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:46.884661 27458 raft_consensus.cc:2454] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:46.884667 27065 raft_consensus.cc:2454] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:46.884951 27264 leader_election.cc:302] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:46.885115 28068 raft_consensus.cc:2781] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:46.885190 28068 raft_consensus.cc:482] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:46.885231 28068 raft_consensus.cc:3037] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.886054 28068 raft_consensus.cc:504] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.886410 28068 leader_election.cc:288] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Requested vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:46.886729 27458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f149fd8790a0412e81ac49d0ce359ede" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:46.886732 27065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f149fd8790a0412e81ac49d0ce359ede" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:46.886830 27458 raft_consensus.cc:3037] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.886833 27065 raft_consensus.cc:3037] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:46.887506 27065 raft_consensus.cc:2454] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:46.887588 27458 raft_consensus.cc:2454] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:46.887791 27262 leader_election.cc:302] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, 3c3f70fdad7b438a841f74fc638d17f5; no voters:
I20240501 19:56:46.887940 28068 raft_consensus.cc:2781] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:46.888020 28068 raft_consensus.cc:686] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:46.888136 28068 consensus_queue.cc:237] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:46.889500 26918 catalog_manager.cc:5484] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } }
W20240501 19:56:47.032013 28282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:47.032227 28282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:47.035224 28282 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240501 19:56:47.103691 27196 raft_consensus.cc:1261] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:47.103890 27328 raft_consensus.cc:1261] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:47.104295 28076 consensus_queue.cc:1035] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:47.104667 28076 consensus_queue.cc:1035] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:47.109438 27197 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:47.109871 27327 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:47.110258 28185 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:47.110702 28278 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:47.112015 27065 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:47.112819 27458 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:47.119298 28057 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:47.119477 28068 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 107.405 ms
time per row: 0.0537026 ms
I20240501 19:56:47.179947 26918 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50992:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:47.185377 27308 tablet_service.cc:1467] Processing CreateTablet for tablet b3e84419a13348e4a0e7bacfe94cf33e (DEFAULT_TABLE table=rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.185648 27308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3e84419a13348e4a0e7bacfe94cf33e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.185639 27177 tablet_service.cc:1467] Processing CreateTablet for tablet b3e84419a13348e4a0e7bacfe94cf33e (DEFAULT_TABLE table=rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.185906 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3e84419a13348e4a0e7bacfe94cf33e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.186512 27175 tablet_service.cc:1467] Processing CreateTablet for tablet b2c944ca787b475294f9c41f6052a7ea (DEFAULT_TABLE table=rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.186720 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2c944ca787b475294f9c41f6052a7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.188159 28337 tablet_bootstrap.cc:492] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:47.188989 28337 tablet_bootstrap.cc:654] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.189059 27308 tablet_service.cc:1467] Processing CreateTablet for tablet 9102f7c8350843c2bd7009ea190860d0 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.189296 27308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9102f7c8350843c2bd7009ea190860d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.189659 27044 tablet_service.cc:1467] Processing CreateTablet for tablet b3e84419a13348e4a0e7bacfe94cf33e (DEFAULT_TABLE table=rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.189903 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3e84419a13348e4a0e7bacfe94cf33e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.190519 27045 tablet_service.cc:1467] Processing CreateTablet for tablet 9102f7c8350843c2bd7009ea190860d0 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.190747 27045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9102f7c8350843c2bd7009ea190860d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.191020 27438 tablet_service.cc:1467] Processing CreateTablet for tablet 9102f7c8350843c2bd7009ea190860d0 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.191193 27046 tablet_service.cc:1467] Processing CreateTablet for tablet b2c944ca787b475294f9c41f6052a7ea (DEFAULT_TABLE table=rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.191251 27438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9102f7c8350843c2bd7009ea190860d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.191390 27046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2c944ca787b475294f9c41f6052a7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.192672 27439 tablet_service.cc:1467] Processing CreateTablet for tablet b2c944ca787b475294f9c41f6052a7ea (DEFAULT_TABLE table=rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.192873 27439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2c944ca787b475294f9c41f6052a7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.193322 28340 tablet_bootstrap.cc:492] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:47.193535 28339 tablet_bootstrap.cc:492] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:47.194160 28336 tablet_bootstrap.cc:492] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:47.194453 28340 tablet_bootstrap.cc:654] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.194762 28337 tablet_bootstrap.cc:492] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:47.194898 28337 ts_tablet_manager.cc:1367] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240501 19:56:47.195020 28336 tablet_bootstrap.cc:654] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.195394 28337 raft_consensus.cc:348] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.195505 28337 raft_consensus.cc:374] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.195542 28337 raft_consensus.cc:729] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.195715 28339 tablet_bootstrap.cc:654] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.195684 28337 consensus_queue.cc:260] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.196007 28337 ts_tablet_manager.cc:1398] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:47.196182 28337 tablet_bootstrap.cc:492] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:47.196799 28339 tablet_bootstrap.cc:492] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:47.196879 28339 ts_tablet_manager.cc:1367] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:47.197008 28337 tablet_bootstrap.cc:654] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.197263 28339 raft_consensus.cc:348] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.197346 28339 raft_consensus.cc:374] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.197379 28339 raft_consensus.cc:729] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.197489 28339 consensus_queue.cc:260] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.197717 28339 ts_tablet_manager.cc:1398] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:47.197889 28339 tablet_bootstrap.cc:492] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:47.197957 28340 tablet_bootstrap.cc:492] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:47.198050 28340 ts_tablet_manager.cc:1367] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240501 19:56:47.198493 28340 raft_consensus.cc:348] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.198593 28340 raft_consensus.cc:374] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.198635 28340 raft_consensus.cc:729] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.198716 28339 tablet_bootstrap.cc:654] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.198772 28340 consensus_queue.cc:260] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.199508 28340 ts_tablet_manager.cc:1398] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:47.199666 28340 tablet_bootstrap.cc:492] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:47.199899 28339 tablet_bootstrap.cc:492] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:47.199963 28339 ts_tablet_manager.cc:1367] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:47.200366 28339 raft_consensus.cc:348] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.200456 28339 raft_consensus.cc:374] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.200496 28339 raft_consensus.cc:729] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.200543 28340 tablet_bootstrap.cc:654] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.200603 28339 consensus_queue.cc:260] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.200865 28339 ts_tablet_manager.cc:1398] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:47.201031 28339 tablet_bootstrap.cc:492] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:47.201555 28336 tablet_bootstrap.cc:492] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:47.201642 28336 ts_tablet_manager.cc:1367] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240501 19:56:47.201989 28339 tablet_bootstrap.cc:654] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.202103 28336 raft_consensus.cc:348] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.202219 28336 raft_consensus.cc:374] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.202277 28336 raft_consensus.cc:729] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.202396 28336 consensus_queue.cc:260] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.202508 28340 tablet_bootstrap.cc:492] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:47.202571 28340 ts_tablet_manager.cc:1367] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240501 19:56:47.202654 28336 ts_tablet_manager.cc:1398] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:47.202802 28336 tablet_bootstrap.cc:492] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:47.202960 28340 raft_consensus.cc:348] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.203042 28340 raft_consensus.cc:374] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.203070 28340 raft_consensus.cc:729] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.203302 28340 consensus_queue.cc:260] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.203598 28336 tablet_bootstrap.cc:654] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.203591 28340 ts_tablet_manager.cc:1398] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:47.204274 28339 tablet_bootstrap.cc:492] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:47.204350 28339 ts_tablet_manager.cc:1367] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 19:56:47.204741 28339 raft_consensus.cc:348] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.204870 28339 raft_consensus.cc:374] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.204913 28339 raft_consensus.cc:729] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.205029 28339 consensus_queue.cc:260] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.205281 28339 ts_tablet_manager.cc:1398] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:47.206008 28336 tablet_bootstrap.cc:492] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:47.206086 28336 ts_tablet_manager.cc:1367] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240501 19:56:47.206681 28336 raft_consensus.cc:348] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.206897 28336 raft_consensus.cc:374] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.206930 28337 tablet_bootstrap.cc:492] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:47.206955 28336 raft_consensus.cc:729] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.207005 28337 ts_tablet_manager.cc:1367] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240501 19:56:47.207070 28336 consensus_queue.cc:260] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.207327 28336 ts_tablet_manager.cc:1398] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:47.207408 28337 raft_consensus.cc:348] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.207525 28337 raft_consensus.cc:374] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.207559 28337 raft_consensus.cc:729] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.207710 28337 consensus_queue.cc:260] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.207990 28337 ts_tablet_manager.cc:1398] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:47.298323 28322 raft_consensus.cc:482] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:47.298415 28322 raft_consensus.cc:504] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.298782 28322 leader_election.cc:288] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:47.299139 27327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3e84419a13348e4a0e7bacfe94cf33e" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5" is_pre_election: true
I20240501 19:56:47.299284 27327 raft_consensus.cc:2454] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:47.299573 27132 leader_election.cc:302] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:47.299751 28322 raft_consensus.cc:2781] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:47.299810 28322 raft_consensus.cc:482] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:47.299854 28322 raft_consensus.cc:3037] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.300729 28322 raft_consensus.cc:504] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.301026 28322 leader_election.cc:288] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:47.301381 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3e84419a13348e4a0e7bacfe94cf33e" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:47.301492 27328 raft_consensus.cc:3037] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.302294 27328 raft_consensus.cc:2454] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 1.
I20240501 19:56:47.302533 27132 leader_election.cc:302] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:47.302511 27065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3e84419a13348e4a0e7bacfe94cf33e" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:47.302568 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3e84419a13348e4a0e7bacfe94cf33e" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:47.302631 27065 raft_consensus.cc:2454] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:47.302774 28322 raft_consensus.cc:2781] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:47.302857 28322 raft_consensus.cc:686] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Becoming Leader. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Running, Role: LEADER
I20240501 19:56:47.302995 28322 consensus_queue.cc:237] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.304461 26918 catalog_manager.cc:5484] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2). New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } }
I20240501 19:56:47.346079 28322 raft_consensus.cc:482] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:47.346180 28322 raft_consensus.cc:504] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.346544 28322 leader_election.cc:288] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:47.346839 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c944ca787b475294f9c41f6052a7ea" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:47.346876 27458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c944ca787b475294f9c41f6052a7ea" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:47.346974 27066 raft_consensus.cc:2454] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:47.347003 27458 raft_consensus.cc:2454] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:47.347249 27131 leader_election.cc:302] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:47.347450 28322 raft_consensus.cc:2781] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:47.347499 28322 raft_consensus.cc:482] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:47.347522 28322 raft_consensus.cc:3037] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.348170 28322 raft_consensus.cc:504] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.348413 28322 leader_election.cc:288] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Requested vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:47.348734 27458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c944ca787b475294f9c41f6052a7ea" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:47.348731 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2c944ca787b475294f9c41f6052a7ea" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:47.348821 27458 raft_consensus.cc:3037] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.348846 27066 raft_consensus.cc:3037] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.349478 27458 raft_consensus.cc:2454] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 1.
I20240501 19:56:47.349532 27066 raft_consensus.cc:2454] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 1.
I20240501 19:56:47.349716 27133 leader_election.cc:302] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5057264da53436ea17bad01c66a7fbf, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:47.349908 28322 raft_consensus.cc:2781] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:47.349992 28322 raft_consensus.cc:686] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Becoming Leader. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Running, Role: LEADER
I20240501 19:56:47.350128 28322 consensus_queue.cc:237] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:47.351483 26918 catalog_manager.cc:5484] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2). New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } }
I20240501 19:56:47.377769 28333 raft_consensus.cc:482] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:47.377856 28333 raft_consensus.cc:504] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.378278 28333 leader_election.cc:288] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:47.378572 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9102f7c8350843c2bd7009ea190860d0" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:47.378666 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9102f7c8350843c2bd7009ea190860d0" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5" is_pre_election: true
I20240501 19:56:47.378695 27066 raft_consensus.cc:2454] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 0.
I20240501 19:56:47.378798 27328 raft_consensus.cc:2454] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 0.
I20240501 19:56:47.378993 27393 leader_election.cc:302] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:47.379129 28333 raft_consensus.cc:2781] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:47.379181 28333 raft_consensus.cc:482] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:47.379218 28333 raft_consensus.cc:3037] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.379943 28333 raft_consensus.cc:504] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.380236 28333 leader_election.cc:288] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 election: Requested vote from peers 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:47.380527 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9102f7c8350843c2bd7009ea190860d0" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:47.380544 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9102f7c8350843c2bd7009ea190860d0" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:47.380630 27066 raft_consensus.cc:3037] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.380630 27328 raft_consensus.cc:3037] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.381407 27328 raft_consensus.cc:2454] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 1.
I20240501 19:56:47.381407 27066 raft_consensus.cc:2454] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 1.
I20240501 19:56:47.381685 27394 leader_election.cc:302] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:47.381839 28333 raft_consensus.cc:2781] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:47.381922 28333 raft_consensus.cc:686] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Becoming Leader. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Running, Role: LEADER
I20240501 19:56:47.382037 28333 consensus_queue.cc:237] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.383446 26918 catalog_manager.cc:5484] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf reported cstate change: term changed from 0 to 1, leader changed from <none> to e5057264da53436ea17bad01c66a7fbf (127.20.210.4). New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } }
W20240501 19:56:47.524685 28353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:47.524989 28353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:47.528039 28353 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240501 19:56:47.608055 27066 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:47.608598 28333 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:47.611617 27065 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:47.612058 28322 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:47.617915 27458 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:47.619478 27328 raft_consensus.cc:1261] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:47.620395 27064 raft_consensus.cc:3037] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.621423 27064 raft_consensus.cc:1261] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:47.622524 28198 consensus_queue.cc:1035] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:47.622820 28198 consensus_queue.cc:1035] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:47.624451 28210 mvcc.cc:205] Tried to move back new op lower bound from 7022974597550858240 to 7022974596637974528. Current Snapshot: MvccSnapshot[applied={T|T < 7022974597550858240}]
I20240501 19:56:47.626669 27327 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:47.627734 28198 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:47.629601 28185 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 111.069 ms
time per row: 0.0555347 ms
I20240501 19:56:47.675506 26922 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50992:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:47.680214 27175 tablet_service.cc:1467] Processing CreateTablet for tablet ae366b11dc9f492b9467d68947cd3dc1 (DEFAULT_TABLE table=rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.680245 27308 tablet_service.cc:1467] Processing CreateTablet for tablet ae366b11dc9f492b9467d68947cd3dc1 (DEFAULT_TABLE table=rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.680500 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae366b11dc9f492b9467d68947cd3dc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.680507 27308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae366b11dc9f492b9467d68947cd3dc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.680994 27046 tablet_service.cc:1467] Processing CreateTablet for tablet a74a8248e5584fb483728f7676502ad5 (DEFAULT_TABLE table=rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.681210 27046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a74a8248e5584fb483728f7676502ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.681221 27307 tablet_service.cc:1467] Processing CreateTablet for tablet 5a9674523a2148b1a6c1a5d22a57aac5 (DEFAULT_TABLE table=rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.681438 27307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a9674523a2148b1a6c1a5d22a57aac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.682580 28336 tablet_bootstrap.cc:492] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:47.682585 28337 tablet_bootstrap.cc:492] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:47.683377 28336 tablet_bootstrap.cc:654] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.683450 28337 tablet_bootstrap.cc:654] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.683435 27306 tablet_service.cc:1467] Processing CreateTablet for tablet a74a8248e5584fb483728f7676502ad5 (DEFAULT_TABLE table=rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.683634 27306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a74a8248e5584fb483728f7676502ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.684320 28337 tablet_bootstrap.cc:492] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:47.684374 28337 ts_tablet_manager.cc:1367] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:47.684643 28337 raft_consensus.cc:348] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.684712 28337 raft_consensus.cc:374] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.684736 28337 raft_consensus.cc:729] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.684844 28337 consensus_queue.cc:260] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.685140 28337 ts_tablet_manager.cc:1398] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:47.685397 28339 tablet_bootstrap.cc:492] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:47.685999 28339 tablet_bootstrap.cc:654] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.686780 27177 tablet_service.cc:1467] Processing CreateTablet for tablet a74a8248e5584fb483728f7676502ad5 (DEFAULT_TABLE table=rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.686983 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a74a8248e5584fb483728f7676502ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.686986 27439 tablet_service.cc:1467] Processing CreateTablet for tablet ae366b11dc9f492b9467d68947cd3dc1 (DEFAULT_TABLE table=rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.687029 27438 tablet_service.cc:1467] Processing CreateTablet for tablet 5a9674523a2148b1a6c1a5d22a57aac5 (DEFAULT_TABLE table=rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.687217 28339 tablet_bootstrap.cc:492] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:47.687218 27439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae366b11dc9f492b9467d68947cd3dc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.687279 28339 ts_tablet_manager.cc:1367] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:47.687333 27438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a9674523a2148b1a6c1a5d22a57aac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.687657 28339 raft_consensus.cc:348] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:47.687757 28339 raft_consensus.cc:374] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.687781 28339 raft_consensus.cc:729] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.687882 28339 consensus_queue.cc:260] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:47.688123 28339 ts_tablet_manager.cc:1398] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:47.688808 28336 tablet_bootstrap.cc:492] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:47.688885 28336 ts_tablet_manager.cc:1367] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240501 19:56:47.689273 28340 tablet_bootstrap.cc:492] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:47.689435 28336 raft_consensus.cc:348] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.689533 28336 raft_consensus.cc:374] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.689565 28336 raft_consensus.cc:729] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.689675 28336 consensus_queue.cc:260] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.689924 28336 ts_tablet_manager.cc:1398] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:47.690025 28336 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:47.690066 28340 tablet_bootstrap.cc:654] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.690167 27175 tablet_service.cc:1467] Processing CreateTablet for tablet 5a9674523a2148b1a6c1a5d22a57aac5 (DEFAULT_TABLE table=rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:47.690395 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a9674523a2148b1a6c1a5d22a57aac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:47.690726 28336 tablet_bootstrap.cc:654] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.692417 28336 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:47.692507 28336 ts_tablet_manager.cc:1367] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240501 19:56:47.692845 28336 raft_consensus.cc:348] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.692956 28336 raft_consensus.cc:374] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.692986 28336 raft_consensus.cc:729] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.693107 28336 consensus_queue.cc:260] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.693271 28340 tablet_bootstrap.cc:492] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:47.693379 28340 ts_tablet_manager.cc:1367] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:56:47.693436 28336 ts_tablet_manager.cc:1398] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:47.693589 28336 tablet_bootstrap.cc:492] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:47.693737 28340 raft_consensus.cc:348] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.693818 28340 raft_consensus.cc:374] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.693838 28340 raft_consensus.cc:729] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.693949 28340 consensus_queue.cc:260] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.694195 28340 ts_tablet_manager.cc:1398] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:47.694759 28337 tablet_bootstrap.cc:492] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:47.694399 28336 tablet_bootstrap.cc:654] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.695468 28340 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:47.695647 28337 tablet_bootstrap.cc:654] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.696393 28340 tablet_bootstrap.cc:654] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.696643 28337 tablet_bootstrap.cc:492] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:47.696718 28337 ts_tablet_manager.cc:1367] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:47.697058 28337 raft_consensus.cc:348] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:47.697175 28337 raft_consensus.cc:374] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.697214 28337 raft_consensus.cc:729] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.697340 28337 consensus_queue.cc:260] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:47.697551 28336 tablet_bootstrap.cc:492] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:47.697569 28337 ts_tablet_manager.cc:1398] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:47.697644 28336 ts_tablet_manager.cc:1367] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240501 19:56:47.697714 28337 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:47.698083 28336 raft_consensus.cc:348] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:47.698189 28336 raft_consensus.cc:374] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.698220 28336 raft_consensus.cc:729] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.698407 28337 tablet_bootstrap.cc:654] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:47.698366 28336 consensus_queue.cc:260] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:47.698614 28336 ts_tablet_manager.cc:1398] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:47.699074 28340 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:47.699162 28340 ts_tablet_manager.cc:1367] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:56:47.699321 28337 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:47.699404 28337 ts_tablet_manager.cc:1367] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:47.699609 28340 raft_consensus.cc:348] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.699702 28340 raft_consensus.cc:374] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.699733 28340 raft_consensus.cc:729] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.699842 28340 consensus_queue.cc:260] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.699808 28337 raft_consensus.cc:348] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.699911 28337 raft_consensus.cc:374] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:47.699951 28337 raft_consensus.cc:729] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:47.700091 28340 ts_tablet_manager.cc:1398] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:47.700102 28337 consensus_queue.cc:260] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.700364 28337 ts_tablet_manager.cc:1398] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:47.708307 28384 raft_consensus.cc:482] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:47.708415 28384 raft_consensus.cc:504] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:47.708758 28384 leader_election.cc:288] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:47.709021 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a74a8248e5584fb483728f7676502ad5" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:47.709029 27327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a74a8248e5584fb483728f7676502ad5" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5" is_pre_election: true
I20240501 19:56:47.709146 27064 raft_consensus.cc:2454] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:47.709149 27327 raft_consensus.cc:2454] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:47.709415 27132 leader_election.cc:302] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:47.709594 28384 raft_consensus.cc:2781] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:47.709646 28384 raft_consensus.cc:482] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:47.709695 28384 raft_consensus.cc:3037] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.710392 28384 raft_consensus.cc:504] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:47.710685 28384 leader_election.cc:288] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Requested vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:47.710939 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a74a8248e5584fb483728f7676502ad5" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:47.710964 27327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a74a8248e5584fb483728f7676502ad5" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:47.711026 27064 raft_consensus.cc:3037] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.711061 27327 raft_consensus.cc:3037] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.711655 27064 raft_consensus.cc:2454] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 1.
I20240501 19:56:47.711670 27327 raft_consensus.cc:2454] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 1.
I20240501 19:56:47.711922 27131 leader_election.cc:302] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:47.712069 28384 raft_consensus.cc:2781] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:47.712157 28384 raft_consensus.cc:686] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Becoming Leader. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Running, Role: LEADER
I20240501 19:56:47.712272 28384 consensus_queue.cc:237] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:47.713563 26921 catalog_manager.cc:5484] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2). New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:47.725934 28068 raft_consensus.cc:482] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:47.726022 28068 raft_consensus.cc:504] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.726411 28068 leader_election.cc:288] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:47.726667 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae366b11dc9f492b9467d68947cd3dc1" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:47.726712 27458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae366b11dc9f492b9467d68947cd3dc1" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:47.726759 27196 raft_consensus.cc:2454] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:47.726819 27458 raft_consensus.cc:2454] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:47.727015 27261 leader_election.cc:302] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:47.727152 28068 raft_consensus.cc:2781] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:47.727200 28068 raft_consensus.cc:482] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:47.727238 28068 raft_consensus.cc:3037] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.727818 28068 raft_consensus.cc:504] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.728081 28068 leader_election.cc:288] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Requested vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:47.728322 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae366b11dc9f492b9467d68947cd3dc1" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:47.728381 27458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae366b11dc9f492b9467d68947cd3dc1" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:47.728415 27197 raft_consensus.cc:3037] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.728467 27458 raft_consensus.cc:3037] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.729193 27458 raft_consensus.cc:2454] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:47.729449 27264 leader_election.cc:302] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:47.729584 28068 raft_consensus.cc:2781] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:47.729646 28068 raft_consensus.cc:686] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:47.729755 28068 consensus_queue.cc:237] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.731031 26921 catalog_manager.cc:5484] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:47.746866 27197 raft_consensus.cc:2454] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:47.928000 28068 raft_consensus.cc:482] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:47.928107 28068 raft_consensus.cc:504] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.928447 28068 leader_election.cc:288] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:47.928790 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9674523a2148b1a6c1a5d22a57aac5" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:47.928834 27458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9674523a2148b1a6c1a5d22a57aac5" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:47.928926 27197 raft_consensus.cc:2454] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:47.929018 27458 raft_consensus.cc:2454] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:47.929230 27261 leader_election.cc:302] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:47.929428 28068 raft_consensus.cc:2781] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:47.929490 28068 raft_consensus.cc:482] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:47.929549 28068 raft_consensus.cc:3037] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.930387 28068 raft_consensus.cc:504] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.930689 28068 leader_election.cc:288] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Requested vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:47.930965 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9674523a2148b1a6c1a5d22a57aac5" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:47.930995 27458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9674523a2148b1a6c1a5d22a57aac5" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:47.931087 27458 raft_consensus.cc:3037] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.931085 27197 raft_consensus.cc:3037] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:47.931916 27458 raft_consensus.cc:2454] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:47.931917 27197 raft_consensus.cc:2454] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:47.932173 27264 leader_election.cc:302] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:47.932329 28068 raft_consensus.cc:2781] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:47.932405 28068 raft_consensus.cc:686] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:47.932552 28068 consensus_queue.cc:237] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:47.933957 26921 catalog_manager.cc:5484] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } }
W20240501 19:56:48.067209 28403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:48.067405 28403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:48.070363 28403 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240501 19:56:48.119127 28384 consensus_queue.cc:1035] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:48.125761 28384 consensus_queue.cc:1035] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:48.146261 28068 consensus_queue.cc:1035] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:48.151261 27458 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:48.151664 28400 consensus_queue.cc:1035] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:48.151801 28068 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:48.152005 27197 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:48.152423 28436 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 121.645 ms
time per row: 0.0608224 ms
I20240501 19:56:48.227468 26918 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50992:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:48.232098 27046 tablet_service.cc:1467] Processing CreateTablet for tablet 82f387c0d03a473fb548b447dc867cc7 (DEFAULT_TABLE table=rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.232208 27438 tablet_service.cc:1467] Processing CreateTablet for tablet 82f387c0d03a473fb548b447dc867cc7 (DEFAULT_TABLE table=rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.232395 27046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82f387c0d03a473fb548b447dc867cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.232461 27438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82f387c0d03a473fb548b447dc867cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.232774 27306 tablet_service.cc:1467] Processing CreateTablet for tablet 0723850372f8416d9b8ac19f119939bc (DEFAULT_TABLE table=rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.233016 27306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0723850372f8416d9b8ac19f119939bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.233268 27439 tablet_service.cc:1467] Processing CreateTablet for tablet 0723850372f8416d9b8ac19f119939bc (DEFAULT_TABLE table=rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.233309 27177 tablet_service.cc:1467] Processing CreateTablet for tablet 3ca822a1462544c3be45371d6e088447 (DEFAULT_TABLE table=rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.233448 27439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0723850372f8416d9b8ac19f119939bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.233546 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca822a1462544c3be45371d6e088447. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.233835 27175 tablet_service.cc:1467] Processing CreateTablet for tablet 82f387c0d03a473fb548b447dc867cc7 (DEFAULT_TABLE table=rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.234042 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82f387c0d03a473fb548b447dc867cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.234557 27437 tablet_service.cc:1467] Processing CreateTablet for tablet 3ca822a1462544c3be45371d6e088447 (DEFAULT_TABLE table=rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.234761 27437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca822a1462544c3be45371d6e088447. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.235486 28454 tablet_bootstrap.cc:492] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:48.235531 27045 tablet_service.cc:1467] Processing CreateTablet for tablet 3ca822a1462544c3be45371d6e088447 (DEFAULT_TABLE table=rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.235769 27045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca822a1462544c3be45371d6e088447. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.235971 28453 tablet_bootstrap.cc:492] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:48.236171 28455 tablet_bootstrap.cc:492] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:48.236387 28454 tablet_bootstrap.cc:654] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.236846 28453 tablet_bootstrap.cc:654] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.236958 28455 tablet_bootstrap.cc:654] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.237422 28454 tablet_bootstrap.cc:492] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:48.237493 28454 ts_tablet_manager.cc:1367] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:48.237802 28453 tablet_bootstrap.cc:492] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:48.237869 28453 ts_tablet_manager.cc:1367] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:48.237892 28454 raft_consensus.cc:348] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.237967 28454 raft_consensus.cc:374] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.237987 28454 raft_consensus.cc:729] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.238094 28454 consensus_queue.cc:260] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.238250 28453 raft_consensus.cc:348] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.238369 28454 ts_tablet_manager.cc:1398] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:48.238389 28453 raft_consensus.cc:374] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.238420 28453 raft_consensus.cc:729] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.238415 27046 tablet_service.cc:1467] Processing CreateTablet for tablet 0723850372f8416d9b8ac19f119939bc (DEFAULT_TABLE table=rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.238544 28453 consensus_queue.cc:260] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.238603 27046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0723850372f8416d9b8ac19f119939bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.238853 28453 ts_tablet_manager.cc:1398] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:48.239010 28453 tablet_bootstrap.cc:492] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:48.239831 28455 tablet_bootstrap.cc:492] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:48.239849 28453 tablet_bootstrap.cc:654] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.240023 28455 ts_tablet_manager.cc:1367] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 19:56:48.240489 28455 raft_consensus.cc:348] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.240609 28455 raft_consensus.cc:374] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.240666 28455 raft_consensus.cc:729] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.240793 28455 consensus_queue.cc:260] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.241108 28455 ts_tablet_manager.cc:1398] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:48.241364 28453 tablet_bootstrap.cc:492] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:48.241454 28453 ts_tablet_manager.cc:1367] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 19:56:48.241843 28453 raft_consensus.cc:348] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.242010 28453 raft_consensus.cc:374] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.241989 28455 tablet_bootstrap.cc:492] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:48.242053 28453 raft_consensus.cc:729] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.242417 28453 consensus_queue.cc:260] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.242738 28453 ts_tablet_manager.cc:1398] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:48.243139 28455 tablet_bootstrap.cc:654] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.244061 28453 tablet_bootstrap.cc:492] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:48.244166 28455 tablet_bootstrap.cc:492] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:48.244251 28455 ts_tablet_manager.cc:1367] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:48.244668 28455 raft_consensus.cc:348] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.244880 28455 raft_consensus.cc:374] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.244920 28453 tablet_bootstrap.cc:654] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.244930 28455 raft_consensus.cc:729] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.245064 28455 consensus_queue.cc:260] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.245842 28455 ts_tablet_manager.cc:1398] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:48.246374 28453 tablet_bootstrap.cc:492] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:48.246454 28453 ts_tablet_manager.cc:1367] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 19:56:48.246814 28453 raft_consensus.cc:348] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.246904 28453 raft_consensus.cc:374] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.246937 28453 raft_consensus.cc:729] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.247073 28453 consensus_queue.cc:260] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.247633 28453 ts_tablet_manager.cc:1398] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:48.259713 28462 tablet_bootstrap.cc:492] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:48.260399 28462 tablet_bootstrap.cc:654] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.261251 28462 tablet_bootstrap.cc:492] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:48.261337 28462 ts_tablet_manager.cc:1367] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:48.261690 28462 raft_consensus.cc:348] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.261765 28462 raft_consensus.cc:374] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.261811 28462 raft_consensus.cc:729] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.261945 28462 consensus_queue.cc:260] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.262135 28462 ts_tablet_manager.cc:1398] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:48.262284 28462 tablet_bootstrap.cc:492] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:48.263195 28462 tablet_bootstrap.cc:654] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.264042 28462 tablet_bootstrap.cc:492] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:48.264129 28462 ts_tablet_manager.cc:1367] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:48.264530 28462 raft_consensus.cc:348] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.264607 28462 raft_consensus.cc:374] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.264631 28462 raft_consensus.cc:729] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.264763 28462 consensus_queue.cc:260] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.264989 28462 ts_tablet_manager.cc:1398] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:48.265172 28462 tablet_bootstrap.cc:492] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:48.265959 28462 tablet_bootstrap.cc:654] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.266970 28462 tablet_bootstrap.cc:492] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:48.267047 28462 ts_tablet_manager.cc:1367] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:48.267416 28462 raft_consensus.cc:348] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.267522 28462 raft_consensus.cc:374] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.267552 28462 raft_consensus.cc:729] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.267647 28462 consensus_queue.cc:260] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.267925 28462 ts_tablet_manager.cc:1398] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:48.282282 28322 raft_consensus.cc:482] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:48.282380 28322 raft_consensus.cc:504] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.282680 28322 leader_election.cc:288] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:48.282972 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82f387c0d03a473fb548b447dc867cc7" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:48.282997 27457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82f387c0d03a473fb548b447dc867cc7" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:48.283092 27457 raft_consensus.cc:2454] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:48.283090 27064 raft_consensus.cc:2454] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:48.283342 27131 leader_election.cc:302] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:48.283507 28322 raft_consensus.cc:2781] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:48.283555 28322 raft_consensus.cc:482] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:48.283579 28322 raft_consensus.cc:3037] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.284276 28322 raft_consensus.cc:504] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.284588 28322 leader_election.cc:288] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Requested vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:48.284847 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82f387c0d03a473fb548b447dc867cc7" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:48.284862 27457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82f387c0d03a473fb548b447dc867cc7" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:48.284935 27457 raft_consensus.cc:3037] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.284945 27064 raft_consensus.cc:3037] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.285590 27457 raft_consensus.cc:2454] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 1.
I20240501 19:56:48.285645 27064 raft_consensus.cc:2454] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 1.
I20240501 19:56:48.285863 27133 leader_election.cc:302] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5057264da53436ea17bad01c66a7fbf, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:48.286020 28322 raft_consensus.cc:2781] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:48.286119 28322 raft_consensus.cc:686] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Becoming Leader. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Running, Role: LEADER
I20240501 19:56:48.286262 28322 consensus_queue.cc:237] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.287740 26919 catalog_manager.cc:5484] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2). New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } }
I20240501 19:56:48.346395 28435 raft_consensus.cc:482] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:48.346485 28435 raft_consensus.cc:504] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.346879 28435 leader_election.cc:288] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:48.347162 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca822a1462544c3be45371d6e088447" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:48.347172 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca822a1462544c3be45371d6e088447" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:48.347285 27064 raft_consensus.cc:2454] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 0.
I20240501 19:56:48.347313 27196 raft_consensus.cc:2454] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 0.
I20240501 19:56:48.347518 27393 leader_election.cc:302] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:48.347697 28435 raft_consensus.cc:2781] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:48.347755 28435 raft_consensus.cc:482] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:48.347791 28435 raft_consensus.cc:3037] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.348699 28435 raft_consensus.cc:504] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.348977 28435 leader_election.cc:288] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 election: Requested vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:48.349228 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca822a1462544c3be45371d6e088447" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:48.349335 27196 raft_consensus.cc:3037] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.350112 27196 raft_consensus.cc:2454] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 1.
I20240501 19:56:48.350333 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ca822a1462544c3be45371d6e088447" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:48.350364 27392 leader_election.cc:302] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5057264da53436ea17bad01c66a7fbf, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:48.350425 27064 raft_consensus.cc:3037] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.350509 28435 raft_consensus.cc:2781] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:48.350580 28435 raft_consensus.cc:686] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Becoming Leader. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Running, Role: LEADER
I20240501 19:56:48.350694 28435 consensus_queue.cc:237] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.351183 27064 raft_consensus.cc:2454] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 1.
I20240501 19:56:48.352219 26919 catalog_manager.cc:5484] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf reported cstate change: term changed from 0 to 1, leader changed from <none> to e5057264da53436ea17bad01c66a7fbf (127.20.210.4). New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:48.399576 28441 raft_consensus.cc:482] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:48.399675 28441 raft_consensus.cc:504] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.400020 28441 leader_election.cc:288] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:48.400346 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0723850372f8416d9b8ac19f119939bc" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:48.400346 27457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0723850372f8416d9b8ac19f119939bc" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:48.400467 27064 raft_consensus.cc:2454] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:48.400467 27457 raft_consensus.cc:2454] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:48.400740 27264 leader_election.cc:302] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:48.400923 28441 raft_consensus.cc:2781] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:48.400980 28441 raft_consensus.cc:482] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:48.401006 28441 raft_consensus.cc:3037] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.401726 28441 raft_consensus.cc:504] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.402042 28441 leader_election.cc:288] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Requested vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:48.402323 27457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0723850372f8416d9b8ac19f119939bc" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:48.402381 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0723850372f8416d9b8ac19f119939bc" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:48.402439 27457 raft_consensus.cc:3037] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.402498 27064 raft_consensus.cc:3037] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.403175 27457 raft_consensus.cc:2454] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:48.403182 27064 raft_consensus.cc:2454] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:48.403456 27262 leader_election.cc:302] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, 3c3f70fdad7b438a841f74fc638d17f5; no voters:
I20240501 19:56:48.403607 28441 raft_consensus.cc:2781] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:48.403681 28441 raft_consensus.cc:686] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:48.403864 28441 consensus_queue.cc:237] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.405599 26919 catalog_manager.cc:5484] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } }
W20240501 19:56:48.517778 28469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:48.517961 28469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:48.520596 28469 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240501 19:56:48.604331 27064 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:48.604743 27457 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:48.605275 28441 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:48.609367 27196 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:48.609598 27459 raft_consensus.cc:1261] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:48.609753 28278 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:48.610260 28322 consensus_queue.cc:1035] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:48.611402 27064 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:48.611754 28278 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:48.615097 28441 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:48.615710 27064 raft_consensus.cc:1261] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:48.616269 28322 consensus_queue.cc:1035] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:48.618988 28438 mvcc.cc:205] Tried to move back new op lower bound from 7022974601660399616 to 7022974600341766144. Current Snapshot: MvccSnapshot[applied={T|T < 7022974601660399616}]
INSERT report
rows total: 2000
time total: 106.561 ms
time per row: 0.0532806 ms
I20240501 19:56:48.667524 26915 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50992:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:48.672122 27046 tablet_service.cc:1467] Processing CreateTablet for tablet 6211064bb3ce41f8b19729d8d0de7fe8 (DEFAULT_TABLE table=rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.672391 27046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6211064bb3ce41f8b19729d8d0de7fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.672449 27045 tablet_service.cc:1467] Processing CreateTablet for tablet 0aaa59df03844aadafb8d57797685dc2 (DEFAULT_TABLE table=rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.672616 27045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aaa59df03844aadafb8d57797685dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.673064 27439 tablet_service.cc:1467] Processing CreateTablet for tablet 0aaa59df03844aadafb8d57797685dc2 (DEFAULT_TABLE table=rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.673134 27177 tablet_service.cc:1467] Processing CreateTablet for tablet 75cc96baf4654318a09dcc34eb32819c (DEFAULT_TABLE table=rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.673341 27439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aaa59df03844aadafb8d57797685dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.673341 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75cc96baf4654318a09dcc34eb32819c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.673722 27306 tablet_service.cc:1467] Processing CreateTablet for tablet 6211064bb3ce41f8b19729d8d0de7fe8 (DEFAULT_TABLE table=rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.673961 27306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6211064bb3ce41f8b19729d8d0de7fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.674682 27175 tablet_service.cc:1467] Processing CreateTablet for tablet 6211064bb3ce41f8b19729d8d0de7fe8 (DEFAULT_TABLE table=rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.674896 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6211064bb3ce41f8b19729d8d0de7fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.675266 28455 tablet_bootstrap.cc:492] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:48.675958 28455 tablet_bootstrap.cc:654] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.676098 27437 tablet_service.cc:1467] Processing CreateTablet for tablet 75cc96baf4654318a09dcc34eb32819c (DEFAULT_TABLE table=rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.676321 27437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75cc96baf4654318a09dcc34eb32819c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.676751 28453 tablet_bootstrap.cc:492] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:48.677564 28453 tablet_bootstrap.cc:654] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.678323 28462 tablet_bootstrap.cc:492] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:48.678576 27306 tablet_service.cc:1467] Processing CreateTablet for tablet 0aaa59df03844aadafb8d57797685dc2 (DEFAULT_TABLE table=rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.678791 27306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aaa59df03844aadafb8d57797685dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.679109 28453 tablet_bootstrap.cc:492] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:48.679140 28462 tablet_bootstrap.cc:654] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.679203 28453 ts_tablet_manager.cc:1367] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:48.679684 28454 tablet_bootstrap.cc:492] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:48.679684 28453 raft_consensus.cc:348] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.679781 28453 raft_consensus.cc:374] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.679818 28453 raft_consensus.cc:729] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.679924 28453 consensus_queue.cc:260] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.680186 28453 ts_tablet_manager.cc:1398] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:48.680505 28454 tablet_bootstrap.cc:654] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.681222 28455 tablet_bootstrap.cc:492] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:48.681316 28455 ts_tablet_manager.cc:1367] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 19:56:48.681291 27307 tablet_service.cc:1467] Processing CreateTablet for tablet 75cc96baf4654318a09dcc34eb32819c (DEFAULT_TABLE table=rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:48.681495 27307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75cc96baf4654318a09dcc34eb32819c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:48.681720 28455 raft_consensus.cc:348] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:48.681818 28455 raft_consensus.cc:374] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.681845 28455 raft_consensus.cc:729] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.681954 28455 consensus_queue.cc:260] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:48.682235 28455 ts_tablet_manager.cc:1398] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:48.682410 28455 tablet_bootstrap.cc:492] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:48.683168 28455 tablet_bootstrap.cc:654] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.684356 28454 tablet_bootstrap.cc:492] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:48.684448 28454 ts_tablet_manager.cc:1367] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 19:56:48.684893 28454 raft_consensus.cc:348] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.684981 28454 raft_consensus.cc:374] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.685029 28454 raft_consensus.cc:729] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.685140 28454 consensus_queue.cc:260] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.685434 28454 ts_tablet_manager.cc:1398] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:48.685602 28454 tablet_bootstrap.cc:492] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:48.686484 28454 tablet_bootstrap.cc:654] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.687443 28453 tablet_bootstrap.cc:492] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:48.687505 28454 tablet_bootstrap.cc:492] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:48.687578 28454 ts_tablet_manager.cc:1367] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:48.688005 28454 raft_consensus.cc:348] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.688087 28455 tablet_bootstrap.cc:492] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:48.688104 28454 raft_consensus.cc:374] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.688138 28454 raft_consensus.cc:729] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.688160 28455 ts_tablet_manager.cc:1367] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240501 19:56:48.688304 28453 tablet_bootstrap.cc:654] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.688252 28454 consensus_queue.cc:260] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.688539 28454 ts_tablet_manager.cc:1398] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:48.688515 28455 raft_consensus.cc:348] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.688596 28455 raft_consensus.cc:374] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.688629 28455 raft_consensus.cc:729] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.688694 28454 tablet_bootstrap.cc:492] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:48.688745 28455 consensus_queue.cc:260] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.688997 28455 ts_tablet_manager.cc:1398] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:48.689821 28454 tablet_bootstrap.cc:654] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.690230 28453 tablet_bootstrap.cc:492] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:48.690637 28453 ts_tablet_manager.cc:1367] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240501 19:56:48.690716 28462 tablet_bootstrap.cc:492] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:48.690809 28462 ts_tablet_manager.cc:1367] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.001s
I20240501 19:56:48.691064 28453 raft_consensus.cc:348] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.691182 28453 raft_consensus.cc:374] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.691220 28453 raft_consensus.cc:729] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.691253 28462 raft_consensus.cc:348] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.691358 28462 raft_consensus.cc:374] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.691390 28462 raft_consensus.cc:729] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.691349 28453 consensus_queue.cc:260] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.691498 28462 consensus_queue.cc:260] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.691610 28453 ts_tablet_manager.cc:1398] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:48.691766 28462 ts_tablet_manager.cc:1398] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:48.691938 28462 tablet_bootstrap.cc:492] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:48.692242 28454 tablet_bootstrap.cc:492] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:48.692328 28454 ts_tablet_manager.cc:1367] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 19:56:48.692771 28462 tablet_bootstrap.cc:654] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:48.692715 28454 raft_consensus.cc:348] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:48.692831 28454 raft_consensus.cc:374] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.692867 28454 raft_consensus.cc:729] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.693006 28454 consensus_queue.cc:260] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:48.693274 28454 ts_tablet_manager.cc:1398] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:48.694077 28462 tablet_bootstrap.cc:492] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:48.694165 28462 ts_tablet_manager.cc:1367] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:48.694583 28462 raft_consensus.cc:348] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:48.694684 28462 raft_consensus.cc:374] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:48.694721 28462 raft_consensus.cc:729] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:48.694856 28462 consensus_queue.cc:260] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:48.695084 28462 ts_tablet_manager.cc:1398] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:48.734272 28400 raft_consensus.cc:482] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:48.734350 28400 raft_consensus.cc:504] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.734642 28400 leader_election.cc:288] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:48.734886 27457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aaa59df03844aadafb8d57797685dc2" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:48.734915 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aaa59df03844aadafb8d57797685dc2" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:48.735002 27064 raft_consensus.cc:2454] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:48.735002 27457 raft_consensus.cc:2454] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:48.735215 27264 leader_election.cc:302] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:48.735383 28068 raft_consensus.cc:2781] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:48.735445 28400 raft_consensus.cc:482] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:48.735450 28068 raft_consensus.cc:482] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:48.735594 28068 raft_consensus.cc:3037] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.735563 28400 raft_consensus.cc:504] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.735893 28400 leader_election.cc:288] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:48.736158 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6211064bb3ce41f8b19729d8d0de7fe8" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:48.736181 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6211064bb3ce41f8b19729d8d0de7fe8" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:48.736300 27064 raft_consensus.cc:2454] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:48.736307 27196 raft_consensus.cc:2454] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:48.736508 28068 raft_consensus.cc:504] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.736541 27262 leader_election.cc:302] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, 3c3f70fdad7b438a841f74fc638d17f5; no voters:
I20240501 19:56:48.736747 28400 raft_consensus.cc:2781] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:48.736805 28400 raft_consensus.cc:482] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:48.736840 28400 raft_consensus.cc:3037] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.736863 28068 leader_election.cc:288] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Requested vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:48.737075 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aaa59df03844aadafb8d57797685dc2" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:48.737177 27064 raft_consensus.cc:3037] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.737563 28400 raft_consensus.cc:504] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.737841 27064 raft_consensus.cc:2454] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:48.737860 28400 leader_election.cc:288] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Requested vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:48.738003 27457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aaa59df03844aadafb8d57797685dc2" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:48.738072 27262 leader_election.cc:302] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, 3c3f70fdad7b438a841f74fc638d17f5; no voters:
I20240501 19:56:48.738128 27457 raft_consensus.cc:3037] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.738214 28400 raft_consensus.cc:2781] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:48.738201 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6211064bb3ce41f8b19729d8d0de7fe8" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:48.738312 28400 raft_consensus.cc:686] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:48.738333 27064 raft_consensus.cc:3037] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.738417 28400 consensus_queue.cc:237] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:48.739030 27064 raft_consensus.cc:2454] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:48.739030 27457 raft_consensus.cc:2454] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:48.739264 27262 leader_election.cc:302] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, 3c3f70fdad7b438a841f74fc638d17f5; no voters:
I20240501 19:56:48.739398 28400 raft_consensus.cc:2781] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:48.739416 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6211064bb3ce41f8b19729d8d0de7fe8" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:48.739511 28400 raft_consensus.cc:686] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:48.739518 27196 raft_consensus.cc:3037] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.739621 28400 consensus_queue.cc:237] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:48.740252 27196 raft_consensus.cc:2454] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:48.741032 26924 catalog_manager.cc:5484] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } }
I20240501 19:56:48.743614 26924 catalog_manager.cc:5484] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:48.903545 28400 raft_consensus.cc:482] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:48.903658 28400 raft_consensus.cc:504] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:48.904062 28400 leader_election.cc:288] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:48.904376 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75cc96baf4654318a09dcc34eb32819c" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:48.904426 27457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75cc96baf4654318a09dcc34eb32819c" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:48.904505 27196 raft_consensus.cc:2454] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:48.904570 27457 raft_consensus.cc:2454] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:48.904767 27261 leader_election.cc:302] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:48.904917 28400 raft_consensus.cc:2781] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:48.904989 28400 raft_consensus.cc:482] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:48.905042 28400 raft_consensus.cc:3037] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.905851 28400 raft_consensus.cc:504] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:48.906155 28400 leader_election.cc:288] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Requested vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:48.906468 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75cc96baf4654318a09dcc34eb32819c" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:48.906510 27457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75cc96baf4654318a09dcc34eb32819c" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:48.906603 27457 raft_consensus.cc:3037] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.906601 27196 raft_consensus.cc:3037] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:48.907480 27196 raft_consensus.cc:2454] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:48.907480 27457 raft_consensus.cc:2454] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:48.907719 27264 leader_election.cc:302] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:48.907872 28400 raft_consensus.cc:2781] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:48.907946 28400 raft_consensus.cc:686] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:48.908066 28400 consensus_queue.cc:237] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } }
I20240501 19:56:48.909703 26924 catalog_manager.cc:5484] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } }
W20240501 19:56:49.016484 28522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:49.016713 28522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:49.019421 28522 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240501 19:56:49.095575 27459 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:49.095878 27457 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:49.095963 27064 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:49.096262 27066 raft_consensus.cc:1261] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:49.096268 28400 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:49.096582 28400 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:49.096632 27196 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:49.097218 28441 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:49.098407 27197 raft_consensus.cc:1261] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240501 19:56:49.098724 28441 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:49.098997 28441 consensus_queue.cc:1035] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:49.099215 28441 consensus_queue.cc:1035] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:49.113698 28445 mvcc.cc:205] Tried to move back new op lower bound from 7022974603649191936 to 7022974602888376320. Current Snapshot: MvccSnapshot[applied={T|T < 7022974603649191936}]
INSERT report
rows total: 2000
time total: 101.658 ms
time per row: 0.0508289 ms
I20240501 19:56:49.158200 26916 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50992:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:49.164068 27046 tablet_service.cc:1467] Processing CreateTablet for tablet 3e05a10bfc6540c19faa154896b8976a (DEFAULT_TABLE table=rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.164192 27307 tablet_service.cc:1467] Processing CreateTablet for tablet 3e05a10bfc6540c19faa154896b8976a (DEFAULT_TABLE table=rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.164361 27046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e05a10bfc6540c19faa154896b8976a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.164479 27307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e05a10bfc6540c19faa154896b8976a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.164630 27175 tablet_service.cc:1467] Processing CreateTablet for tablet 3e05a10bfc6540c19faa154896b8976a (DEFAULT_TABLE table=rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.164891 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e05a10bfc6540c19faa154896b8976a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.166842 28454 tablet_bootstrap.cc:492] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:49.166903 28455 tablet_bootstrap.cc:492] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:49.166992 28453 tablet_bootstrap.cc:492] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:49.167336 27046 tablet_service.cc:1467] Processing CreateTablet for tablet e379c346ae524bd28f6e550051c0bf7e (DEFAULT_TABLE table=rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.167567 27046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e379c346ae524bd28f6e550051c0bf7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.167670 28454 tablet_bootstrap.cc:654] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.167704 28453 tablet_bootstrap.cc:654] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.167738 28455 tablet_bootstrap.cc:654] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.168375 27045 tablet_service.cc:1467] Processing CreateTablet for tablet 6aff1d7fd2d2476095ef0a0529110b49 (DEFAULT_TABLE table=rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.168429 27437 tablet_service.cc:1467] Processing CreateTablet for tablet e379c346ae524bd28f6e550051c0bf7e (DEFAULT_TABLE table=rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.168568 27045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aff1d7fd2d2476095ef0a0529110b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.168635 27437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e379c346ae524bd28f6e550051c0bf7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.170491 28462 tablet_bootstrap.cc:492] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:49.170915 28455 tablet_bootstrap.cc:492] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:49.170987 28455 ts_tablet_manager.cc:1367] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240501 19:56:49.171319 28462 tablet_bootstrap.cc:654] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.171391 28455 raft_consensus.cc:348] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.171486 28455 raft_consensus.cc:374] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.171516 28455 raft_consensus.cc:729] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.171633 28455 consensus_queue.cc:260] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.171914 28455 ts_tablet_manager.cc:1398] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:49.172256 27437 tablet_service.cc:1467] Processing CreateTablet for tablet 6aff1d7fd2d2476095ef0a0529110b49 (DEFAULT_TABLE table=rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.172467 27437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aff1d7fd2d2476095ef0a0529110b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.173027 27307 tablet_service.cc:1467] Processing CreateTablet for tablet 6aff1d7fd2d2476095ef0a0529110b49 (DEFAULT_TABLE table=rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.173243 27307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aff1d7fd2d2476095ef0a0529110b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.173650 27175 tablet_service.cc:1467] Processing CreateTablet for tablet e379c346ae524bd28f6e550051c0bf7e (DEFAULT_TABLE table=rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.173852 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e379c346ae524bd28f6e550051c0bf7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.175575 28455 tablet_bootstrap.cc:492] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:49.176311 28462 tablet_bootstrap.cc:492] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:49.176393 28462 ts_tablet_manager.cc:1367] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 19:56:49.176422 28455 tablet_bootstrap.cc:654] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.176867 28462 raft_consensus.cc:348] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.176957 28462 raft_consensus.cc:374] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.176985 28462 raft_consensus.cc:729] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.177088 28462 consensus_queue.cc:260] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.177341 28462 ts_tablet_manager.cc:1398] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:49.177455 28455 tablet_bootstrap.cc:492] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:49.177501 28462 tablet_bootstrap.cc:492] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:49.177523 28455 ts_tablet_manager.cc:1367] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 19:56:49.177899 28455 raft_consensus.cc:348] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.178007 28455 raft_consensus.cc:374] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.178037 28455 raft_consensus.cc:729] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.178100 28462 tablet_bootstrap.cc:654] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.178145 28455 consensus_queue.cc:260] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.178900 28454 tablet_bootstrap.cc:492] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:49.178972 28454 ts_tablet_manager.cc:1367] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240501 19:56:49.178550 28455 ts_tablet_manager.cc:1398] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:49.179337 28454 raft_consensus.cc:348] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.179417 28454 raft_consensus.cc:374] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.179443 28454 raft_consensus.cc:729] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.179549 28454 consensus_queue.cc:260] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.179791 28454 ts_tablet_manager.cc:1398] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:49.180585 28454 tablet_bootstrap.cc:492] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:49.181574 28454 tablet_bootstrap.cc:654] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.183580 28462 tablet_bootstrap.cc:492] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:49.183655 28462 ts_tablet_manager.cc:1367] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240501 19:56:49.183876 28453 tablet_bootstrap.cc:492] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:49.183944 28453 ts_tablet_manager.cc:1367] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240501 19:56:49.184135 28462 raft_consensus.cc:348] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.184317 28453 raft_consensus.cc:348] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.184401 28453 raft_consensus.cc:374] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.184424 28453 raft_consensus.cc:729] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.184518 28453 consensus_queue.cc:260] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.184228 28462 raft_consensus.cc:374] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.184728 28453 ts_tablet_manager.cc:1398] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:49.184751 28462 raft_consensus.cc:729] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.184863 28453 tablet_bootstrap.cc:492] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:49.184935 28454 tablet_bootstrap.cc:492] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:49.185014 28454 ts_tablet_manager.cc:1367] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:56:49.185287 28462 consensus_queue.cc:260] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.185364 28454 raft_consensus.cc:348] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.185434 28454 raft_consensus.cc:374] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.185468 28454 raft_consensus.cc:729] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.185571 28462 ts_tablet_manager.cc:1398] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:56:49.185608 28453 tablet_bootstrap.cc:654] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.185606 28454 consensus_queue.cc:260] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.185856 28454 ts_tablet_manager.cc:1398] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:49.191124 28453 tablet_bootstrap.cc:492] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:49.191200 28453 ts_tablet_manager.cc:1367] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 19:56:49.191521 28453 raft_consensus.cc:348] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.191603 28453 raft_consensus.cc:374] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.191639 28453 raft_consensus.cc:729] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.191744 28453 consensus_queue.cc:260] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.191989 28453 ts_tablet_manager.cc:1398] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:49.192194 28453 tablet_bootstrap.cc:492] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:49.192883 28453 tablet_bootstrap.cc:654] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.194785 28453 tablet_bootstrap.cc:492] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:49.194870 28453 ts_tablet_manager.cc:1367] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:49.195204 28453 raft_consensus.cc:348] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.195276 28453 raft_consensus.cc:374] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.195295 28453 raft_consensus.cc:729] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.195412 28453 consensus_queue.cc:260] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.195683 28453 ts_tablet_manager.cc:1398] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:49.211119 28547 raft_consensus.cc:482] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:49.211201 28547 raft_consensus.cc:504] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.211575 28547 leader_election.cc:288] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:49.211833 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e05a10bfc6540c19faa154896b8976a" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:49.211869 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e05a10bfc6540c19faa154896b8976a" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:49.211941 27197 raft_consensus.cc:2454] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:49.211977 27066 raft_consensus.cc:2454] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 0.
I20240501 19:56:49.212185 27261 leader_election.cc:302] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:49.212334 28547 raft_consensus.cc:2781] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:49.212388 28547 raft_consensus.cc:482] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:49.212435 28547 raft_consensus.cc:3037] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.213207 28547 raft_consensus.cc:504] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.213482 28547 leader_election.cc:288] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Requested vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:49.213783 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e05a10bfc6540c19faa154896b8976a" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:49.213783 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e05a10bfc6540c19faa154896b8976a" candidate_uuid: "3c3f70fdad7b438a841f74fc638d17f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:49.213887 27066 raft_consensus.cc:3037] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.213888 27197 raft_consensus.cc:3037] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.214687 27066 raft_consensus.cc:2454] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:49.214687 27197 raft_consensus.cc:2454] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3f70fdad7b438a841f74fc638d17f5 in term 1.
I20240501 19:56:49.214972 27261 leader_election.cc:302] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:49.215109 28547 raft_consensus.cc:2781] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:49.215184 28547 raft_consensus.cc:686] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:49.215288 28547 consensus_queue.cc:237] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.216910 26915 catalog_manager.cc:5484] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } }
I20240501 19:56:49.247843 28278 raft_consensus.cc:482] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:49.247936 28278 raft_consensus.cc:504] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.248272 28278 leader_election.cc:288] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:49.248538 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e379c346ae524bd28f6e550051c0bf7e" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:49.248557 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e379c346ae524bd28f6e550051c0bf7e" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:49.248657 27066 raft_consensus.cc:2454] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 0.
I20240501 19:56:49.248657 27197 raft_consensus.cc:2454] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 0.
I20240501 19:56:49.248906 27392 leader_election.cc:302] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5057264da53436ea17bad01c66a7fbf, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:49.249064 28435 raft_consensus.cc:2781] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:49.249114 28435 raft_consensus.cc:482] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:49.249146 28435 raft_consensus.cc:3037] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.250005 28435 raft_consensus.cc:504] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.250339 28435 leader_election.cc:288] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 election: Requested vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:49.250569 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e379c346ae524bd28f6e550051c0bf7e" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:49.250624 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e379c346ae524bd28f6e550051c0bf7e" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:49.250669 27197 raft_consensus.cc:3037] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.250715 27064 raft_consensus.cc:3037] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.251291 27197 raft_consensus.cc:2454] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 1.
I20240501 19:56:49.251358 27064 raft_consensus.cc:2454] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 1.
I20240501 19:56:49.251565 27392 leader_election.cc:302] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5057264da53436ea17bad01c66a7fbf, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:49.251708 28435 raft_consensus.cc:2781] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:49.251777 28435 raft_consensus.cc:686] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Becoming Leader. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Running, Role: LEADER
I20240501 19:56:49.251905 28435 consensus_queue.cc:237] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.253276 26915 catalog_manager.cc:5484] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf reported cstate change: term changed from 0 to 1, leader changed from <none> to e5057264da53436ea17bad01c66a7fbf (127.20.210.4). New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:49.365032 28428 raft_consensus.cc:482] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:49.365129 28428 raft_consensus.cc:504] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.365509 28428 leader_election.cc:288] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:49.365828 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aff1d7fd2d2476095ef0a0529110b49" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:49.365854 27327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aff1d7fd2d2476095ef0a0529110b49" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5" is_pre_election: true
I20240501 19:56:49.365950 27459 raft_consensus.cc:2454] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 0.
I20240501 19:56:49.365976 27327 raft_consensus.cc:2454] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 0.
I20240501 19:56:49.366223 27002 leader_election.cc:302] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:49.366461 28428 raft_consensus.cc:2781] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:49.366509 28428 raft_consensus.cc:482] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:49.366554 28428 raft_consensus.cc:3037] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.367332 28428 raft_consensus.cc:504] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.367630 28428 leader_election.cc:288] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 election: Requested vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:49.367945 27327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aff1d7fd2d2476095ef0a0529110b49" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:49.367951 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aff1d7fd2d2476095ef0a0529110b49" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:49.368050 27327 raft_consensus.cc:3037] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.368050 27459 raft_consensus.cc:3037] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.368913 27327 raft_consensus.cc:2454] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 1.
I20240501 19:56:49.368913 27459 raft_consensus.cc:2454] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 1.
I20240501 19:56:49.369169 27001 leader_election.cc:302] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, 3c3f70fdad7b438a841f74fc638d17f5; no voters:
I20240501 19:56:49.369375 28428 raft_consensus.cc:2781] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:49.369550 28428 raft_consensus.cc:686] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Becoming Leader. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Running, Role: LEADER
I20240501 19:56:49.369675 28428 consensus_queue.cc:237] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } }
I20240501 19:56:49.371315 26915 catalog_manager.cc:5484] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: term changed from 0 to 1, leader changed from <none> to 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1). New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } }
W20240501 19:56:49.511758 28569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:49.511970 28569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:49.514781 28569 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240501 19:56:49.587707 27064 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:49.587790 27197 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:49.588128 28555 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:49.588260 28547 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:49.595705 27459 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:49.596127 28428 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:49.596747 27327 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:49.597132 28428 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:49.598615 28316 mvcc.cc:205] Tried to move back new op lower bound from 7022974605667364864 to 7022974604146991104. Current Snapshot: MvccSnapshot[applied={T|T < 7022974605667364864}]
I20240501 19:56:49.601635 27066 raft_consensus.cc:1261] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:49.601724 27197 raft_consensus.cc:1261] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:49.602392 28435 consensus_queue.cc:1035] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:49.606563 28435 consensus_queue.cc:1035] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 108.006 ms
time per row: 0.0540031 ms
I20240501 19:56:49.660761 26924 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50992:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:56:49.665364 27437 tablet_service.cc:1467] Processing CreateTablet for tablet 896072a285b649dbb31f9f02666ace25 (DEFAULT_TABLE table=rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.665454 27175 tablet_service.cc:1467] Processing CreateTablet for tablet 896072a285b649dbb31f9f02666ace25 (DEFAULT_TABLE table=rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.665683 27437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896072a285b649dbb31f9f02666ace25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.665737 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896072a285b649dbb31f9f02666ace25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.665858 27307 tablet_service.cc:1467] Processing CreateTablet for tablet 896072a285b649dbb31f9f02666ace25 (DEFAULT_TABLE table=rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.666096 27307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896072a285b649dbb31f9f02666ace25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.667476 28462 tablet_bootstrap.cc:492] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:49.667961 27306 tablet_service.cc:1467] Processing CreateTablet for tablet 360f349692784ae69f01ddb9afe73da4 (DEFAULT_TABLE table=rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.668049 28454 tablet_bootstrap.cc:492] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:49.668167 27306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360f349692784ae69f01ddb9afe73da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.668298 28462 tablet_bootstrap.cc:654] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.668766 28454 tablet_bootstrap.cc:654] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.669704 27177 tablet_service.cc:1467] Processing CreateTablet for tablet 55d39ed3702046b08b1ac3c76c272c01 (DEFAULT_TABLE table=rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.669847 27045 tablet_service.cc:1467] Processing CreateTablet for tablet 55d39ed3702046b08b1ac3c76c272c01 (DEFAULT_TABLE table=rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.669925 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d39ed3702046b08b1ac3c76c272c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.669945 28455 tablet_bootstrap.cc:492] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:49.670073 27045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d39ed3702046b08b1ac3c76c272c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.670431 28454 tablet_bootstrap.cc:492] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:49.670492 28454 ts_tablet_manager.cc:1367] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:49.670713 28455 tablet_bootstrap.cc:654] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.670965 28454 raft_consensus.cc:348] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.671056 28454 raft_consensus.cc:374] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.671087 28454 raft_consensus.cc:729] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.671200 28454 consensus_queue.cc:260] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.671466 28454 ts_tablet_manager.cc:1398] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:49.671598 28454 tablet_bootstrap.cc:492] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:49.672089 28453 tablet_bootstrap.cc:492] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e: Bootstrap starting.
I20240501 19:56:49.672094 27437 tablet_service.cc:1467] Processing CreateTablet for tablet 360f349692784ae69f01ddb9afe73da4 (DEFAULT_TABLE table=rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.672286 27437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360f349692784ae69f01ddb9afe73da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.672300 28454 tablet_bootstrap.cc:654] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.672820 28453 tablet_bootstrap.cc:654] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.674082 28453 tablet_bootstrap.cc:492] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e: No bootstrap required, opened a new log
I20240501 19:56:49.674153 28453 ts_tablet_manager.cc:1367] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:49.674211 27307 tablet_service.cc:1467] Processing CreateTablet for tablet 55d39ed3702046b08b1ac3c76c272c01 (DEFAULT_TABLE table=rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.674450 27307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d39ed3702046b08b1ac3c76c272c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.674561 28462 tablet_bootstrap.cc:492] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:49.674623 28462 ts_tablet_manager.cc:1367] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240501 19:56:49.674875 28453 raft_consensus.cc:348] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.674963 28453 raft_consensus.cc:374] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.674993 28453 raft_consensus.cc:729] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.675052 28462 raft_consensus.cc:348] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.675094 28453 consensus_queue.cc:260] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.675132 28462 raft_consensus.cc:374] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.675159 28462 raft_consensus.cc:729] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.675253 28462 consensus_queue.cc:260] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.675343 28453 ts_tablet_manager.cc:1398] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:49.675500 28462 ts_tablet_manager.cc:1398] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:49.676342 28462 tablet_bootstrap.cc:492] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf: Bootstrap starting.
I20240501 19:56:49.676458 27175 tablet_service.cc:1467] Processing CreateTablet for tablet 360f349692784ae69f01ddb9afe73da4 (DEFAULT_TABLE table=rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:56:49.676657 27175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360f349692784ae69f01ddb9afe73da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:49.677192 28462 tablet_bootstrap.cc:654] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.678138 28462 tablet_bootstrap.cc:492] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf: No bootstrap required, opened a new log
I20240501 19:56:49.678275 28462 ts_tablet_manager.cc:1367] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:49.678690 28454 tablet_bootstrap.cc:492] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:49.678767 28454 ts_tablet_manager.cc:1367] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240501 19:56:49.679114 28454 raft_consensus.cc:348] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.679181 28454 raft_consensus.cc:374] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.679198 28454 raft_consensus.cc:729] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.679260 28454 consensus_queue.cc:260] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.679486 28454 ts_tablet_manager.cc:1398] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:49.679628 28454 tablet_bootstrap.cc:492] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5: Bootstrap starting.
I20240501 19:56:49.679997 28455 tablet_bootstrap.cc:492] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:49.680068 28455 ts_tablet_manager.cc:1367] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240501 19:56:49.680416 28455 raft_consensus.cc:348] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.680501 28455 raft_consensus.cc:374] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.680529 28455 raft_consensus.cc:729] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.680534 28454 tablet_bootstrap.cc:654] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.680632 28455 consensus_queue.cc:260] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.680416 28462 raft_consensus.cc:348] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.680805 28462 raft_consensus.cc:374] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.680845 28462 raft_consensus.cc:729] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.680904 28455 ts_tablet_manager.cc:1398] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:49.680940 28462 consensus_queue.cc:260] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.681039 28455 tablet_bootstrap.cc:492] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:49.681581 28462 ts_tablet_manager.cc:1398] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:49.681768 28455 tablet_bootstrap.cc:654] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.682467 28454 tablet_bootstrap.cc:492] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5: No bootstrap required, opened a new log
I20240501 19:56:49.682534 28454 ts_tablet_manager.cc:1367] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:49.682904 28454 raft_consensus.cc:348] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.683015 28454 raft_consensus.cc:374] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.683063 28454 raft_consensus.cc:729] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.683166 28454 consensus_queue.cc:260] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.683421 28454 ts_tablet_manager.cc:1398] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:56:49.683516 28455 tablet_bootstrap.cc:492] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:49.683596 28455 ts_tablet_manager.cc:1367] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:56:49.684119 28455 raft_consensus.cc:348] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.684219 28455 raft_consensus.cc:374] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.684271 28455 raft_consensus.cc:729] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.684393 28455 consensus_queue.cc:260] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.684679 28455 ts_tablet_manager.cc:1398] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:49.684839 28455 tablet_bootstrap.cc:492] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6: Bootstrap starting.
I20240501 19:56:49.685608 28455 tablet_bootstrap.cc:654] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6: Neither blocks nor log segments found. Creating new log.
I20240501 19:56:49.686581 28455 tablet_bootstrap.cc:492] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6: No bootstrap required, opened a new log
I20240501 19:56:49.686667 28455 ts_tablet_manager.cc:1367] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:56:49.687089 28455 raft_consensus.cc:348] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.687191 28455 raft_consensus.cc:374] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:56:49.687230 28455 raft_consensus.cc:729] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Initialized, Role: FOLLOWER
I20240501 19:56:49.687350 28455 consensus_queue.cc:260] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.687570 28455 ts_tablet_manager.cc:1398] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:49.774075 28435 raft_consensus.cc:482] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:49.774175 28435 raft_consensus.cc:504] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.774534 28435 leader_election.cc:288] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:49.774833 27195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360f349692784ae69f01ddb9afe73da4" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" is_pre_election: true
I20240501 19:56:49.774818 27327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360f349692784ae69f01ddb9afe73da4" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5" is_pre_election: true
I20240501 19:56:49.774971 27327 raft_consensus.cc:2454] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 0.
I20240501 19:56:49.774971 27195 raft_consensus.cc:2454] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 0.
I20240501 19:56:49.775307 27394 leader_election.cc:302] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:49.775468 28435 raft_consensus.cc:2781] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:49.775586 28435 raft_consensus.cc:482] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:49.775614 28435 raft_consensus.cc:3037] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.776422 28435 raft_consensus.cc:504] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.776741 28435 leader_election.cc:288] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 election: Requested vote from peers 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:49.777001 27327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360f349692784ae69f01ddb9afe73da4" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:49.777014 27195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360f349692784ae69f01ddb9afe73da4" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:49.777089 27327 raft_consensus.cc:3037] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.777096 27195 raft_consensus.cc:3037] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.777815 27195 raft_consensus.cc:2454] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 1.
I20240501 19:56:49.777823 27327 raft_consensus.cc:2454] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 1.
I20240501 19:56:49.778057 27394 leader_election.cc:302] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:56:49.778204 28435 raft_consensus.cc:2781] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:49.778328 28435 raft_consensus.cc:686] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Becoming Leader. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Running, Role: LEADER
I20240501 19:56:49.778479 28435 consensus_queue.cc:237] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.779996 26924 catalog_manager.cc:5484] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf reported cstate change: term changed from 0 to 1, leader changed from <none> to e5057264da53436ea17bad01c66a7fbf (127.20.210.4). New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:49.847268 28617 raft_consensus.cc:482] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:49.847368 28617 raft_consensus.cc:504] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.847712 28617 leader_election.cc:288] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:49.847982 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896072a285b649dbb31f9f02666ace25" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf" is_pre_election: true
I20240501 19:56:49.848079 27327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896072a285b649dbb31f9f02666ace25" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5" is_pre_election: true
I20240501 19:56:49.848126 27459 raft_consensus.cc:2454] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:49.848196 27327 raft_consensus.cc:2454] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:49.848414 27133 leader_election.cc:302] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5057264da53436ea17bad01c66a7fbf, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:49.848577 28617 raft_consensus.cc:2781] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:49.848639 28617 raft_consensus.cc:482] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:49.848662 28617 raft_consensus.cc:3037] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.849413 28617 raft_consensus.cc:504] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.849670 28617 leader_election.cc:288] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Requested vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:49.849997 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896072a285b649dbb31f9f02666ace25" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:49.849988 27327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896072a285b649dbb31f9f02666ace25" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:49.850100 27459 raft_consensus.cc:3037] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.850100 27327 raft_consensus.cc:3037] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.851024 27327 raft_consensus.cc:2454] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 1.
I20240501 19:56:49.851024 27459 raft_consensus.cc:2454] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 1.
I20240501 19:56:49.851271 27133 leader_election.cc:302] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5057264da53436ea17bad01c66a7fbf, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:49.851441 28617 raft_consensus.cc:2781] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:49.851541 28617 raft_consensus.cc:686] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Becoming Leader. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Running, Role: LEADER
I20240501 19:56:49.851641 28617 consensus_queue.cc:237] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } }
I20240501 19:56:49.853261 26924 catalog_manager.cc:5484] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2). New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:49.872395 28617 raft_consensus.cc:482] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:56:49.872478 28617 raft_consensus.cc:504] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.872754 28617 leader_election.cc:288] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:49.873122 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d39ed3702046b08b1ac3c76c272c01" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" is_pre_election: true
I20240501 19:56:49.873148 27327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d39ed3702046b08b1ac3c76c272c01" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5" is_pre_election: true
I20240501 19:56:49.873256 27327 raft_consensus.cc:2454] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:49.873257 27064 raft_consensus.cc:2454] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 0.
I20240501 19:56:49.873522 27132 leader_election.cc:302] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:49.873683 28617 raft_consensus.cc:2781] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:56:49.873741 28617 raft_consensus.cc:482] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:56:49.873804 28617 raft_consensus.cc:3037] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.874537 28617 raft_consensus.cc:504] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.874850 28617 leader_election.cc:288] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Requested vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:49.875088 27064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d39ed3702046b08b1ac3c76c272c01" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:49.875133 27327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d39ed3702046b08b1ac3c76c272c01" candidate_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:49.875195 27064 raft_consensus.cc:3037] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.875244 27327 raft_consensus.cc:3037] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:56:49.875936 27064 raft_consensus.cc:2454] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 1.
I20240501 19:56:49.875946 27327 raft_consensus.cc:2454] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef2d33d636574e8fbb9d3b90a15035d6 in term 1.
I20240501 19:56:49.876201 27131 leader_election.cc:302] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:49.876402 28617 raft_consensus.cc:2781] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:56:49.876482 28617 raft_consensus.cc:686] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Becoming Leader. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Running, Role: LEADER
I20240501 19:56:49.876603 28617 consensus_queue.cc:237] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } }
I20240501 19:56:49.878155 26924 catalog_manager.cc:5484] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2). New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } }
W20240501 19:56:49.999845 28623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:50.000056 28623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:50.002822 28623 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240501 19:56:50.092537 27327 raft_consensus.cc:1261] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:50.092566 27195 raft_consensus.cc:1261] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:50.092909 28278 consensus_queue.cc:1035] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:50.093577 27064 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:50.094048 28617 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:50.094115 28278 consensus_queue.cc:1035] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:50.094815 27328 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:50.094899 27459 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:50.095338 27327 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:56:50.098311 28383 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:50.100623 28617 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:56:50.100886 28617 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 116.746 ms
time per row: 0.0583728 ms
I20240501 19:56:50.156906 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.5:42343
--local_ip_for_outbound_sockets=127.20.210.5
--tserver_master_addrs=127.20.210.62:43635
--webserver_port=34519
--webserver_interface=127.20.210.5
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240501 19:56:50.259756 28664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:50.259934 28664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:50.259974 28664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:50.262463 28664 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:50.262511 28664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:50.262566 28664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.5
I20240501 19:56:50.265671 28664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.5:42343
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.20.210.5
--webserver_port=34519
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.20.210.62:43635
--never_fsync=true
--heap_profile_path=/tmp/kudu.28664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.5
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:50.266337 28664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:50.267235 28664 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:50.275123 28669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:50.275703 28670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:50.276461 28672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:50.277232 28664 server_base.cc:1034] running on GCE node
I20240501 19:56:50.277545 28664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:50.277961 28664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:50.279130 28664 hybrid_clock.cc:648] HybridClock initialized: now 1714593410279120 us; error 22 us; skew 500 ppm
I20240501 19:56:50.280663 28664 webserver.cc:460] Webserver started at http://127.20.210.5:34519/ using document root <none> and password file <none>
I20240501 19:56:50.281128 28664 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:50.281196 28664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:50.284251 28664 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 19:56:50.286000 28678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:50.286928 28664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:50.287012 28664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "724bae28280242c6982c4f8e315b0d0b"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:50.287330 28664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:50.303009 28664 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:50.303448 28664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:50.303556 28664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:50.304020 28664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:50.304771 28664 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:50.304824 28664 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:50.304864 28664 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:50.304893 28664 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:50.312350 28664 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.5:42343
I20240501 19:56:50.312381 28791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.5:42343 every 8 connection(s)
I20240501 19:56:50.313272 28664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 19:56:50.320225 28792 heartbeater.cc:347] Connected to a master server at 127.20.210.62:43635
I20240501 19:56:50.320410 28792 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:50.320813 28792 heartbeater.cc:511] Master 127.20.210.62:43635 requested a full tablet report, sending...
I20240501 19:56:50.321655 26924 ts_manager.cc:194] Re-registered known tserver with Master: 724bae28280242c6982c4f8e315b0d0b (127.20.210.5:42343)
I20240501 19:56:50.322161 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 28664
I20240501 19:56:50.322211 26924 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.5:52493
I20240501 19:56:50.322551 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.6:38529
--local_ip_for_outbound_sockets=127.20.210.6
--tserver_master_addrs=127.20.210.62:43635
--webserver_port=45193
--webserver_interface=127.20.210.6
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240501 19:56:50.426462 28795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:50.426645 28795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:50.426687 28795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:50.429396 28795 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:50.429452 28795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:50.429512 28795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.6
I20240501 19:56:50.432750 28795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.6:38529
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.20.210.6
--webserver_port=45193
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.20.210.62:43635
--never_fsync=true
--heap_profile_path=/tmp/kudu.28795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.6
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:50.433468 28795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:50.434445 28795 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:50.441910 28801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:50.442406 28803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:50.443248 28800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:50.443993 28795 server_base.cc:1034] running on GCE node
I20240501 19:56:50.444314 28795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:50.444792 28795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:50.445930 28795 hybrid_clock.cc:648] HybridClock initialized: now 1714593410445910 us; error 33 us; skew 500 ppm
I20240501 19:56:50.451586 28795 webserver.cc:460] Webserver started at http://127.20.210.6:45193/ using document root <none> and password file <none>
I20240501 19:56:50.452045 28795 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:50.452117 28795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:50.455022 28795 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 19:56:50.456825 28809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:50.457793 28795 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:50.457892 28795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "7ccb4fe66ab942d399551b10f5ed32ea"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:50.458292 28795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:50.475265 28795 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:50.475754 28795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:50.475880 28795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:50.476296 28795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:50.477048 28795 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:50.477097 28795 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:50.477133 28795 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:50.477154 28795 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:50.485688 28795 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.6:38529
I20240501 19:56:50.485740 28922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.6:38529 every 8 connection(s)
I20240501 19:56:50.486778 28795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240501 19:56:50.488479 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 28795
I20240501 19:56:50.488945 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.7:33695
--local_ip_for_outbound_sockets=127.20.210.7
--tserver_master_addrs=127.20.210.62:43635
--webserver_port=36437
--webserver_interface=127.20.210.7
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 19:56:50.496484 28923 heartbeater.cc:347] Connected to a master server at 127.20.210.62:43635
I20240501 19:56:50.496721 28923 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:50.497412 28923 heartbeater.cc:511] Master 127.20.210.62:43635 requested a full tablet report, sending...
I20240501 19:56:50.498548 26924 ts_manager.cc:194] Re-registered known tserver with Master: 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529)
I20240501 19:56:50.499192 26924 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.6:45887
W20240501 19:56:50.591665 28926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:50.591884 28926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:50.591939 28926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:50.595007 28926 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:50.595098 28926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:50.595180 28926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.7
I20240501 19:56:50.598765 28926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.7:33695
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.20.210.7
--webserver_port=36437
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.20.210.62:43635
--never_fsync=true
--heap_profile_path=/tmp/kudu.28926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.7
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:50.599432 28926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:50.600379 28926 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:50.607959 28934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:50.608511 28931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:50.609267 28932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:50.610002 28926 server_base.cc:1034] running on GCE node
I20240501 19:56:50.610358 28926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:50.610821 28926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:50.611955 28926 hybrid_clock.cc:648] HybridClock initialized: now 1714593410611935 us; error 33 us; skew 500 ppm
I20240501 19:56:50.613595 28926 webserver.cc:460] Webserver started at http://127.20.210.7:36437/ using document root <none> and password file <none>
I20240501 19:56:50.614112 28926 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:50.614202 28926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:50.617214 28926 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20240501 19:56:50.618945 28940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:50.619764 28926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240501 19:56:50.619859 28926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20"
format_stamp: "Formatted at 2024-05-01 19:56:40 on dist-test-slave-vgh4"
I20240501 19:56:50.620218 28926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:50.641651 28926 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:50.642169 28926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:50.642340 28926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:50.642781 28926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:50.643606 28926 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:50.643651 28926 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:50.643684 28926 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:50.643714 28926 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:50.651800 28926 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.7:33695
I20240501 19:56:50.651893 29053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.7:33695 every 8 connection(s)
I20240501 19:56:50.652673 28926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 19:56:50.655977 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 28926
I20240501 19:56:50.656407 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.8:44887
--local_ip_for_outbound_sockets=127.20.210.8
--tserver_master_addrs=127.20.210.62:43635
--webserver_port=41901
--webserver_interface=127.20.210.8
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 19:56:50.660390 29054 heartbeater.cc:347] Connected to a master server at 127.20.210.62:43635
I20240501 19:56:50.660590 29054 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:50.661144 29054 heartbeater.cc:511] Master 127.20.210.62:43635 requested a full tablet report, sending...
I20240501 19:56:50.662099 26924 ts_manager.cc:194] Re-registered known tserver with Master: ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695)
I20240501 19:56:50.662602 26924 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.7:52231
W20240501 19:56:50.754370 29057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:50.754534 29057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:50.754578 29057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:50.757266 29057 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:50.757320 29057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:50.757409 29057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.8
I20240501 19:56:50.760309 29057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.8:44887
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.20.210.8
--webserver_port=41901
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.20.210.62:43635
--never_fsync=true
--heap_profile_path=/tmp/kudu.29057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.8
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:50.760973 29057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:50.761893 29057 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:56:50.769423 29066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:50.769968 29063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:56:50.770939 29064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:50.771600 29057 server_base.cc:1034] running on GCE node
I20240501 19:56:50.771929 29057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:50.772393 29057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:50.773542 29057 hybrid_clock.cc:648] HybridClock initialized: now 1714593410773520 us; error 37 us; skew 500 ppm
I20240501 19:56:50.775167 29057 webserver.cc:460] Webserver started at http://127.20.210.8:41901/ using document root <none> and password file <none>
I20240501 19:56:50.775624 29057 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:50.775707 29057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:50.778630 29057 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 19:56:50.780261 29072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:50.781093 29057 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:56:50.781176 29057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "474eb4491dcb47f4a2a2f6e149bc39c1"
format_stamp: "Formatted at 2024-05-01 19:56:41 on dist-test-slave-vgh4"
I20240501 19:56:50.781534 29057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:50.797077 29057 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:50.797528 29057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:50.797653 29057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:50.798060 29057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:50.798810 29057 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:50.798853 29057 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:50.798898 29057 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:50.798928 29057 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:50.806450 29057 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.8:44887
I20240501 19:56:50.806478 29185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.8:44887 every 8 connection(s)
I20240501 19:56:50.807353 29057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 19:56:50.811395 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 29057
I20240501 19:56:50.815801 29186 heartbeater.cc:347] Connected to a master server at 127.20.210.62:43635
I20240501 19:56:50.815997 29186 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:50.816434 29186 heartbeater.cc:511] Master 127.20.210.62:43635 requested a full tablet report, sending...
I20240501 19:56:50.817382 26924 ts_manager.cc:194] Re-registered known tserver with Master: 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8:44887)
I20240501 19:56:50.818114 26924 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.8:55799
W20240501 19:56:50.916260 29190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:50.916429 29190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:50.919663 29190 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 19:56:51.035094 27437 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:56:51.035989 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:56:51.038340 28988 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.042726 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.047696 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.050498 29120 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.054003 27175 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 19:56:51.066063 27307 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240501 19:56:51.068163 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.074338 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.083494 29120 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.084362 27307 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240501 19:56:51.090013 28988 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.092595 27437 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:56:51.095114 27175 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 19:56:51.126444 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:56:51.246817 27307 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240501 19:56:51.249050 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.251405 27437 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:56:51.254470 28988 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.258677 29120 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.277526 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.282215 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:56:51.293313 27175 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 19:56:51.324383 28792 heartbeater.cc:503] Master 127.20.210.62:43635 was elected leader, sending a full tablet report...
W20240501 19:56:51.347604 29308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:51.347822 29308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:51.351187 29308 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 19:56:51.418447 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:56:51.422420 27307 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240501 19:56:51.440516 27175 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 19:56:51.440675 27437 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:56:51.448522 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.449944 28988 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.466925 29120 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.469416 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:56:51.470252 29120 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.471841 28988 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.475265 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.479521 27437 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:56:51.485244 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.499701 27307 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240501 19:56:51.501108 28923 heartbeater.cc:503] Master 127.20.210.62:43635 was elected leader, sending a full tablet report...
I20240501 19:56:51.507624 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.524773 27175 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 19:56:51.564047 27326 consensus_queue.cc:237] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:51.565464 27064 raft_consensus.cc:1261] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.565618 27196 raft_consensus.cc:1261] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.566018 28068 consensus_queue.cc:1035] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.566411 28068 consensus_queue.cc:1035] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.568542 29433 raft_consensus.cc:2932] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:51.568852 27196 raft_consensus.cc:2932] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:51.569181 27064 raft_consensus.cc:2932] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
W20240501 19:56:51.569751 27263 consensus_peers.cc:485] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695): Couldn't send request to peer ecdc0d100f5f4b95b9dda10c2f97cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6211064bb3ce41f8b19729d8d0de7fe8. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.571569 26924 catalog_manager.cc:5484] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:51.581506 27459 consensus_queue.cc:237] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:51.583312 27326 raft_consensus.cc:1261] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.583649 27196 raft_consensus.cc:1261] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.583904 29340 consensus_queue.cc:1035] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.584069 28659 consensus_queue.cc:1035] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.586059 29340 raft_consensus.cc:2932] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.586673 27326 raft_consensus.cc:2932] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.589509 26920 catalog_manager.cc:5484] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:51.590198 27196 raft_consensus.cc:2932] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
W20240501 19:56:51.591449 27393 consensus_peers.cc:485] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf -> Peer 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529): Couldn't send request to peer 7ccb4fe66ab942d399551b10f5ed32ea. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360f349692784ae69f01ddb9afe73da4. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.597689 27326 consensus_queue.cc:237] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:51.599139 27459 raft_consensus.cc:1261] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.599534 28068 consensus_queue.cc:1035] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.600972 27064 raft_consensus.cc:1261] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.602144 29433 consensus_queue.cc:1035] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 19:56:51.602553 27263 consensus_peers.cc:485] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8:44887): Couldn't send request to peer 474eb4491dcb47f4a2a2f6e149bc39c1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ecba34e358c41d9b812113c955b6b29. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.602687 29415 raft_consensus.cc:2932] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.603307 27066 raft_consensus.cc:2932] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.604537 27459 raft_consensus.cc:2932] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.606382 26920 catalog_manager.cc:5484] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.619323 27196 consensus_queue.cc:237] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:51.620505 27066 raft_consensus.cc:1261] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.620860 29291 consensus_queue.cc:1035] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.620996 27459 raft_consensus.cc:1261] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.621439 29291 consensus_queue.cc:1035] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.622716 29291 raft_consensus.cc:2932] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
W20240501 19:56:51.624953 27132 consensus_peers.cc:485] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 -> Peer 724bae28280242c6982c4f8e315b0d0b (127.20.210.5:42343): Couldn't send request to peer 724bae28280242c6982c4f8e315b0d0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 82f387c0d03a473fb548b447dc867cc7. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.625169 27459 raft_consensus.cc:2932] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:51.625385 26924 catalog_manager.cc:5484] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:51.626814 27066 raft_consensus.cc:2932] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:51.635627 27196 consensus_queue.cc:237] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:51.636968 27066 raft_consensus.cc:1261] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.637295 28617 consensus_queue.cc:1035] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.637475 27326 raft_consensus.cc:1261] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.638320 28617 consensus_queue.cc:1035] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.638818 28617 raft_consensus.cc:2932] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:51.640631 27326 raft_consensus.cc:2932] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:51.641180 27066 raft_consensus.cc:2932] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
W20240501 19:56:51.642228 27132 consensus_peers.cc:485] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 -> Peer ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695): Couldn't send request to peer ecdc0d100f5f4b95b9dda10c2f97cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a74a8248e5584fb483728f7676502ad5. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.643041 26924 catalog_manager.cc:5484] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:51.651876 27066 consensus_queue.cc:237] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:51.653403 27196 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.653702 27459 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.654018 29341 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.654526 29341 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 19:56:51.657132 29049 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.20.210.7:33695
W20240501 19:56:51.658085 27000 consensus_peers.cc:485] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e -> Peer 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529): Couldn't send request to peer 7ccb4fe66ab942d399551b10f5ed32ea. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 199f9529870242f599358995c19250b2. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.658946 28607 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.659343 27459 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.662200 26920 catalog_manager.cc:5484] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.663961 27196 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.665280 29054 heartbeater.cc:503] Master 127.20.210.62:43635 was elected leader, sending a full tablet report...
I20240501 19:56:51.675508 27326 consensus_queue.cc:237] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:51.676963 27196 raft_consensus.cc:1261] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 19:56:51.677140 27263 consensus_peers.cc:485] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8:44887): Couldn't send request to peer 474eb4491dcb47f4a2a2f6e149bc39c1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3c80c1dcbcd3439ba75066259557e716. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.677291 29433 consensus_queue.cc:1035] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.677350 27066 raft_consensus.cc:1261] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.677657 29433 consensus_queue.cc:1035] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.678777 28068 raft_consensus.cc:2932] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.680554 27196 raft_consensus.cc:2932] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.680866 26920 catalog_manager.cc:5484] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:51.687645 27066 raft_consensus.cc:2932] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.692374 27066 consensus_queue.cc:237] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
W20240501 19:56:51.693579 29053 acceptor_pool.cc:285] unable to collect diagnostics on RPC socket 127.20.210.7:33695: Invalid argument: socket address is ambiguous
I20240501 19:56:51.693661 27459 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.693857 27196 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.694036 28607 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.694223 29484 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.696002 29484 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:51.697793 27459 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:51.697808 27196 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
W20240501 19:56:51.698997 27001 consensus_peers.cc:485] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e -> Peer ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695): Couldn't send request to peer ecdc0d100f5f4b95b9dda10c2f97cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c498eacc5e947bfb01dfb90c7039df0. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.700343 26920 catalog_manager.cc:5484] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
W20240501 19:56:51.706421 29446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:51.706663 29446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240501 19:56:51.708246 27196 consensus_queue.cc:237] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:51.709419 27326 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.709494 27459 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.709733 29291 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.709853 29466 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 19:56:51.709971 29446 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20240501 19:56:51.710357 27132 consensus_peers.cc:485] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 -> Peer 724bae28280242c6982c4f8e315b0d0b (127.20.210.5:42343): Couldn't send request to peer 724bae28280242c6982c4f8e315b0d0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0f417f4001bc48158187ed262f7737ab. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.711501 28617 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:51.711911 27459 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:51.711946 27326 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:51.714305 26924 catalog_manager.cc:5484] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:51.721932 27066 consensus_queue.cc:237] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:51.723122 27196 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.723203 27459 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.723443 29341 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 19:56:51.723620 27000 consensus_peers.cc:485] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e -> Peer 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529): Couldn't send request to peer 7ccb4fe66ab942d399551b10f5ed32ea. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37288cafed764ba18fcc07b6e3c7a722. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.723659 28607 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.725107 29484 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.725538 27196 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.725575 27459 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.728433 26916 catalog_manager.cc:5484] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.736299 27459 consensus_queue.cc:237] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:51.737669 27326 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.738066 27066 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.738420 29451 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.738587 28659 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.740676 28659 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.741300 27326 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
W20240501 19:56:51.741775 27394 consensus_peers.cc:485] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf -> Peer 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8:44887): Couldn't send request to peer 474eb4491dcb47f4a2a2f6e149bc39c1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9102f7c8350843c2bd7009ea190860d0. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.743034 27066 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.743782 26924 catalog_manager.cc:5484] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:51.753360 27326 consensus_queue.cc:237] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:51.754491 27459 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.754823 29415 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.755098 27066 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 19:56:51.755280 27263 consensus_peers.cc:485] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695): Couldn't send request to peer ecdc0d100f5f4b95b9dda10c2f97cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f149fd8790a0412e81ac49d0ce359ede. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.755458 29415 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.756817 28068 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:51.758865 27066 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:51.758968 27459 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:51.760777 26916 catalog_manager.cc:5484] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:51.772570 27326 consensus_queue.cc:237] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:51.773964 27459 raft_consensus.cc:1261] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.773988 27196 raft_consensus.cc:1261] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.774627 29415 consensus_queue.cc:1035] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.774818 28068 consensus_queue.cc:1035] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.777000 28068 raft_consensus.cc:2932] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
W20240501 19:56:51.778522 27263 consensus_peers.cc:485] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer 724bae28280242c6982c4f8e315b0d0b (127.20.210.5:42343): Couldn't send request to peer 724bae28280242c6982c4f8e315b0d0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ae366b11dc9f492b9467d68947cd3dc1. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.778708 27196 raft_consensus.cc:2932] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:51.780890 26915 catalog_manager.cc:5484] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:51.782056 27459 raft_consensus.cc:2932] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:51.792030 27066 consensus_queue.cc:237] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:51.793501 27196 raft_consensus.cc:1261] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.793664 27326 raft_consensus.cc:1261] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.793850 29484 consensus_queue.cc:1035] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.794900 29484 consensus_queue.cc:1035] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 19:56:51.795397 27000 consensus_peers.cc:485] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e -> Peer 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529): Couldn't send request to peer 7ccb4fe66ab942d399551b10f5ed32ea. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c157f7f72dc4211b256eee0015eed23. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.797044 28607 raft_consensus.cc:2932] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.797631 27326 raft_consensus.cc:2932] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.797703 27196 raft_consensus.cc:2932] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.800124 26915 catalog_manager.cc:5484] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.814271 27459 consensus_queue.cc:237] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:51.816385 27196 raft_consensus.cc:1261] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.816910 27066 raft_consensus.cc:1261] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.817119 28659 consensus_queue.cc:1035] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.817319 29451 consensus_queue.cc:1035] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.819273 28659 raft_consensus.cc:2932] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
W20240501 19:56:51.819535 27394 consensus_peers.cc:485] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf -> Peer 724bae28280242c6982c4f8e315b0d0b (127.20.210.5:42343): Couldn't send request to peer 724bae28280242c6982c4f8e315b0d0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e379c346ae524bd28f6e550051c0bf7e. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.819728 29186 heartbeater.cc:503] Master 127.20.210.62:43635 was elected leader, sending a full tablet report...
I20240501 19:56:51.819727 27066 raft_consensus.cc:2932] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:51.821348 27196 raft_consensus.cc:2932] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:51.822445 26920 catalog_manager.cc:5484] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:51.831679 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:56:51.832653 27066 consensus_queue.cc:237] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:51.833477 29120 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.834233 27459 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.834658 28607 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.837127 27326 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.837878 28607 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.838858 28607 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.840598 27459 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.842865 26918 catalog_manager.cc:5484] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 19:56:51.843122 27001 consensus_peers.cc:485] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e -> Peer 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8:44887): Couldn't send request to peer 474eb4491dcb47f4a2a2f6e149bc39c1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6aff1d7fd2d2476095ef0a0529110b49. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.844319 27175 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 19:56:51.848382 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.848742 27437 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:56:51.849340 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.849550 27307 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240501 19:56:51.858850 27327 consensus_queue.cc:237] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
W20240501 19:56:51.860538 27263 consensus_peers.cc:485] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695): Couldn't send request to peer ecdc0d100f5f4b95b9dda10c2f97cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e3b87a633efe4400931f1e0a566c8f14. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.860747 27196 raft_consensus.cc:1261] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.861078 28068 consensus_queue.cc:1035] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.862119 27459 raft_consensus.cc:1261] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.862419 28068 consensus_queue.cc:1035] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.869750 27326 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.871330 28068 raft_consensus.cc:2932] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:51.871695 27459 raft_consensus.cc:2932] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:51.872864 27196 raft_consensus.cc:2932] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:51.873903 26915 catalog_manager.cc:5484] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:51.878227 28988 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:51.899762 27196 consensus_queue.cc:237] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:51.900992 27459 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.901077 27326 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.901438 29466 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.901882 29466 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.903654 29466 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.904826 27326 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
W20240501 19:56:51.911326 27132 consensus_peers.cc:485] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 -> Peer 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8:44887): Couldn't send request to peer 474eb4491dcb47f4a2a2f6e149bc39c1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 896072a285b649dbb31f9f02666ace25. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.912182 27459 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:51.912586 26920 catalog_manager.cc:5484] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
Runtime error: ksck discovered errors
I20240501 19:56:51.919767 27196 consensus_queue.cc:237] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:51.921059 27459 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.921272 27066 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:51.921708 28617 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.921872 29466 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:51.923844 29291 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.925813 27459 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:51.926576 26915 catalog_manager.cc:5484] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:51.928886 27066 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
W20240501 19:56:51.930476 27131 consensus_peers.cc:485] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 -> Peer 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529): Couldn't send request to peer 7ccb4fe66ab942d399551b10f5ed32ea. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2c944ca787b475294f9c41f6052a7ea. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:51.988888 29692 ts_tablet_manager.cc:902] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea: Initiating tablet copy from peer e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:51.991034 29692 tablet_copy_client.cc:308] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:37115
I20240501 19:56:52.001955 27479 tablet_copy_service.cc:140] P e5057264da53436ea17bad01c66a7fbf: Received BeginTabletCopySession request for tablet 360f349692784ae69f01ddb9afe73da4 from peer 7ccb4fe66ab942d399551b10f5ed32ea ({username='slave'} at 127.20.210.6:53755)
I20240501 19:56:52.002077 27479 tablet_copy_service.cc:161] P e5057264da53436ea17bad01c66a7fbf: Beginning new tablet copy session on tablet 360f349692784ae69f01ddb9afe73da4 from peer 7ccb4fe66ab942d399551b10f5ed32ea at {username='slave'} at 127.20.210.6:53755: session id = 7ccb4fe66ab942d399551b10f5ed32ea-360f349692784ae69f01ddb9afe73da4
I20240501 19:56:52.003270 27479 tablet_copy_source_session.cc:205] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.006513 29692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360f349692784ae69f01ddb9afe73da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.012178 29692 tablet_copy_client.cc:783] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.012362 29692 tablet_copy_client.cc:647] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.013613 29693 tablet_copy_client.cc:1002] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.021582 29723 ts_tablet_manager.cc:902] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:52.022270 29723 tablet_copy_client.cc:308] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:52.026852 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 6211064bb3ce41f8b19729d8d0de7fe8 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 ({username='slave'} at 127.20.210.7:35965)
I20240501 19:56:52.027005 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 6211064bb3ce41f8b19729d8d0de7fe8 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 at {username='slave'} at 127.20.210.7:35965: session id = ecdc0d100f5f4b95b9dda10c2f97cf20-6211064bb3ce41f8b19729d8d0de7fe8
I20240501 19:56:52.028264 27348 tablet_copy_source_session.cc:205] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.031517 29723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6211064bb3ce41f8b19729d8d0de7fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.037318 29723 tablet_copy_client.cc:783] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.037508 29723 tablet_copy_client.cc:647] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.038988 29725 tablet_copy_client.cc:1002] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Injecting latency into file download: 1500
W20240501 19:56:52.058931 29646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:52.059095 29646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:52.061686 29646 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240501 19:56:52.071164 29754 ts_tablet_manager.cc:902] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b: Initiating tablet copy from peer ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:52.071794 29754 tablet_copy_client.cc:308] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:43113
I20240501 19:56:52.075222 27217 tablet_copy_service.cc:140] P ef2d33d636574e8fbb9d3b90a15035d6: Received BeginTabletCopySession request for tablet 82f387c0d03a473fb548b447dc867cc7 from peer 724bae28280242c6982c4f8e315b0d0b ({username='slave'} at 127.20.210.5:41023)
I20240501 19:56:52.075373 27217 tablet_copy_service.cc:161] P ef2d33d636574e8fbb9d3b90a15035d6: Beginning new tablet copy session on tablet 82f387c0d03a473fb548b447dc867cc7 from peer 724bae28280242c6982c4f8e315b0d0b at {username='slave'} at 127.20.210.5:41023: session id = 724bae28280242c6982c4f8e315b0d0b-82f387c0d03a473fb548b447dc867cc7
I20240501 19:56:52.076627 27217 tablet_copy_source_session.cc:205] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.079561 29754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82f387c0d03a473fb548b447dc867cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.084812 29754 tablet_copy_client.cc:783] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.085043 29754 tablet_copy_client.cc:647] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.086460 29758 tablet_copy_client.cc:1002] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.086532 29760 ts_tablet_manager.cc:902] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20: Initiating tablet copy from peer 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:52.087042 29760 tablet_copy_client.cc:308] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:33375
I20240501 19:56:52.089988 27086 tablet_copy_service.cc:140] P 16cfc5a5082742c5b09d35e5c4f7f10e: Received BeginTabletCopySession request for tablet 6c498eacc5e947bfb01dfb90c7039df0 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 ({username='slave'} at 127.20.210.7:45013)
I20240501 19:56:52.090097 27086 tablet_copy_service.cc:161] P 16cfc5a5082742c5b09d35e5c4f7f10e: Beginning new tablet copy session on tablet 6c498eacc5e947bfb01dfb90c7039df0 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 at {username='slave'} at 127.20.210.7:45013: session id = ecdc0d100f5f4b95b9dda10c2f97cf20-6c498eacc5e947bfb01dfb90c7039df0
I20240501 19:56:52.091113 27086 tablet_copy_source_session.cc:205] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.092134 29760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c498eacc5e947bfb01dfb90c7039df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.094098 29760 tablet_copy_client.cc:783] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.094251 29760 tablet_copy_client.cc:647] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.095410 29764 tablet_copy_client.cc:1002] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.104552 29775 ts_tablet_manager.cc:902] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea: Initiating tablet copy from peer 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:52.105159 29775 tablet_copy_client.cc:308] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:33375
I20240501 19:56:52.108686 27086 tablet_copy_service.cc:140] P 16cfc5a5082742c5b09d35e5c4f7f10e: Received BeginTabletCopySession request for tablet 199f9529870242f599358995c19250b2 from peer 7ccb4fe66ab942d399551b10f5ed32ea ({username='slave'} at 127.20.210.6:37711)
I20240501 19:56:52.108767 27086 tablet_copy_service.cc:161] P 16cfc5a5082742c5b09d35e5c4f7f10e: Beginning new tablet copy session on tablet 199f9529870242f599358995c19250b2 from peer 7ccb4fe66ab942d399551b10f5ed32ea at {username='slave'} at 127.20.210.6:37711: session id = 7ccb4fe66ab942d399551b10f5ed32ea-199f9529870242f599358995c19250b2
I20240501 19:56:52.109530 27086 tablet_copy_source_session.cc:205] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.110388 29775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199f9529870242f599358995c19250b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.112819 29775 tablet_copy_client.cc:783] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.113008 29775 tablet_copy_client.cc:647] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.113286 29790 ts_tablet_manager.cc:902] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea: Initiating tablet copy from peer 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:52.113736 29790 tablet_copy_client.cc:308] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:33375
I20240501 19:56:52.114092 27085 tablet_copy_service.cc:140] P 16cfc5a5082742c5b09d35e5c4f7f10e: Received BeginTabletCopySession request for tablet 37288cafed764ba18fcc07b6e3c7a722 from peer 7ccb4fe66ab942d399551b10f5ed32ea ({username='slave'} at 127.20.210.6:37711)
I20240501 19:56:52.114176 27085 tablet_copy_service.cc:161] P 16cfc5a5082742c5b09d35e5c4f7f10e: Beginning new tablet copy session on tablet 37288cafed764ba18fcc07b6e3c7a722 from peer 7ccb4fe66ab942d399551b10f5ed32ea at {username='slave'} at 127.20.210.6:37711: session id = 7ccb4fe66ab942d399551b10f5ed32ea-37288cafed764ba18fcc07b6e3c7a722
I20240501 19:56:52.114478 29777 tablet_copy_client.cc:1002] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.114953 27085 tablet_copy_source_session.cc:205] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.116078 29790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37288cafed764ba18fcc07b6e3c7a722. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.117825 29790 tablet_copy_client.cc:783] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.117969 29790 tablet_copy_client.cc:647] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.119042 29792 tablet_copy_client.cc:1002] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.132540 29795 ts_tablet_manager.cc:902] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20: Initiating tablet copy from peer ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:52.132997 29795 tablet_copy_client.cc:308] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:43113
I20240501 19:56:52.140110 27217 tablet_copy_service.cc:140] P ef2d33d636574e8fbb9d3b90a15035d6: Received BeginTabletCopySession request for tablet a74a8248e5584fb483728f7676502ad5 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 ({username='slave'} at 127.20.210.7:46547)
I20240501 19:56:52.140206 27217 tablet_copy_service.cc:161] P ef2d33d636574e8fbb9d3b90a15035d6: Beginning new tablet copy session on tablet a74a8248e5584fb483728f7676502ad5 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 at {username='slave'} at 127.20.210.7:46547: session id = ecdc0d100f5f4b95b9dda10c2f97cf20-a74a8248e5584fb483728f7676502ad5
I20240501 19:56:52.141132 27217 tablet_copy_source_session.cc:205] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.142076 29795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a74a8248e5584fb483728f7676502ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.143704 29795 tablet_copy_client.cc:783] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.143888 29795 tablet_copy_client.cc:647] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.144829 29799 tablet_copy_client.cc:1002] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.156844 27437 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:56:52.159842 29120 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:52.159999 27307 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240501 19:56:52.160619 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:52.171715 28988 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:52.183318 27175 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240501 19:56:52.184612 29811 ts_tablet_manager.cc:902] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:52.185535 29811 tablet_copy_client.cc:308] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:52.189361 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet ae366b11dc9f492b9467d68947cd3dc1 from peer 724bae28280242c6982c4f8e315b0d0b ({username='slave'} at 127.20.210.5:40583)
I20240501 19:56:52.189453 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet ae366b11dc9f492b9467d68947cd3dc1 from peer 724bae28280242c6982c4f8e315b0d0b at {username='slave'} at 127.20.210.5:40583: session id = 724bae28280242c6982c4f8e315b0d0b-ae366b11dc9f492b9467d68947cd3dc1
I20240501 19:56:52.190315 27348 tablet_copy_source_session.cc:205] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.191118 29811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae366b11dc9f492b9467d68947cd3dc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.192817 29811 tablet_copy_client.cc:783] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.192960 29811 tablet_copy_client.cc:647] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.193086 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:52.194350 29812 tablet_copy_client.cc:1002] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.210078 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:56:52.212292 29817 ts_tablet_manager.cc:902] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:52.212477 29816 ts_tablet_manager.cc:902] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:52.213009 29816 tablet_copy_client.cc:308] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:52.213042 29817 tablet_copy_client.cc:308] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:52.215368 29827 ts_tablet_manager.cc:902] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b: Initiating tablet copy from peer ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:52.215873 29827 tablet_copy_client.cc:308] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:43113
I20240501 19:56:52.216271 27217 tablet_copy_service.cc:140] P ef2d33d636574e8fbb9d3b90a15035d6: Received BeginTabletCopySession request for tablet 0f417f4001bc48158187ed262f7737ab from peer 724bae28280242c6982c4f8e315b0d0b ({username='slave'} at 127.20.210.5:41023)
I20240501 19:56:52.216387 27217 tablet_copy_service.cc:161] P ef2d33d636574e8fbb9d3b90a15035d6: Beginning new tablet copy session on tablet 0f417f4001bc48158187ed262f7737ab from peer 724bae28280242c6982c4f8e315b0d0b at {username='slave'} at 127.20.210.5:41023: session id = 724bae28280242c6982c4f8e315b0d0b-0f417f4001bc48158187ed262f7737ab
I20240501 19:56:52.217293 27217 tablet_copy_source_session.cc:205] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.217758 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 5ecba34e358c41d9b812113c955b6b29 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 ({username='slave'} at 127.20.210.8:56643)
I20240501 19:56:52.217875 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 5ecba34e358c41d9b812113c955b6b29 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 at {username='slave'} at 127.20.210.8:56643: session id = 474eb4491dcb47f4a2a2f6e149bc39c1-5ecba34e358c41d9b812113c955b6b29
I20240501 19:56:52.217904 27347 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 3c80c1dcbcd3439ba75066259557e716 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 ({username='slave'} at 127.20.210.8:56643)
I20240501 19:56:52.217998 27347 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 3c80c1dcbcd3439ba75066259557e716 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 at {username='slave'} at 127.20.210.8:56643: session id = 474eb4491dcb47f4a2a2f6e149bc39c1-3c80c1dcbcd3439ba75066259557e716
I20240501 19:56:52.218434 29827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f417f4001bc48158187ed262f7737ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.218768 27348 tablet_copy_source_session.cc:205] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.218886 27347 tablet_copy_source_session.cc:205] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.220875 29827 tablet_copy_client.cc:783] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.221037 29827 tablet_copy_client.cc:647] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.222023 29828 tablet_copy_client.cc:1002] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.222260 29816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecba34e358c41d9b812113c955b6b29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.225875 29817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c80c1dcbcd3439ba75066259557e716. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.228801 29817 tablet_copy_client.cc:783] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.228859 29816 tablet_copy_client.cc:783] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.228963 29816 tablet_copy_client.cc:647] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.229007 29817 tablet_copy_client.cc:647] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.230083 29819 tablet_copy_client.cc:1002] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.230147 29823 tablet_copy_client.cc:1002] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Injecting latency into file download: 1500
Runtime error: ksck discovered errors
I20240501 19:56:52.249897 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.9:0
--local_ip_for_outbound_sockets=127.20.210.9
--webserver_interface=127.20.210.9
--webserver_port=0
--tserver_master_addrs=127.20.210.62:43635
--builtin_ntp_servers=127.20.210.20:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240501 19:56:52.252142 29843 ts_tablet_manager.cc:902] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:52.252633 29843 tablet_copy_client.cc:308] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:52.253070 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet f149fd8790a0412e81ac49d0ce359ede from peer ecdc0d100f5f4b95b9dda10c2f97cf20 ({username='slave'} at 127.20.210.7:35965)
I20240501 19:56:52.253155 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet f149fd8790a0412e81ac49d0ce359ede from peer ecdc0d100f5f4b95b9dda10c2f97cf20 at {username='slave'} at 127.20.210.7:35965: session id = ecdc0d100f5f4b95b9dda10c2f97cf20-f149fd8790a0412e81ac49d0ce359ede
I20240501 19:56:52.254066 27348 tablet_copy_source_session.cc:205] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.255373 29843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f149fd8790a0412e81ac49d0ce359ede. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.256899 29843 tablet_copy_client.cc:783] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.257061 29843 tablet_copy_client.cc:647] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.258008 29846 tablet_copy_client.cc:1002] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.300674 29888 ts_tablet_manager.cc:902] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea: Initiating tablet copy from peer 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:52.301160 29888 tablet_copy_client.cc:308] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:33375
I20240501 19:56:52.301762 27085 tablet_copy_service.cc:140] P 16cfc5a5082742c5b09d35e5c4f7f10e: Received BeginTabletCopySession request for tablet 8c157f7f72dc4211b256eee0015eed23 from peer 7ccb4fe66ab942d399551b10f5ed32ea ({username='slave'} at 127.20.210.6:37711)
I20240501 19:56:52.301860 27085 tablet_copy_service.cc:161] P 16cfc5a5082742c5b09d35e5c4f7f10e: Beginning new tablet copy session on tablet 8c157f7f72dc4211b256eee0015eed23 from peer 7ccb4fe66ab942d399551b10f5ed32ea at {username='slave'} at 127.20.210.6:37711: session id = 7ccb4fe66ab942d399551b10f5ed32ea-8c157f7f72dc4211b256eee0015eed23
I20240501 19:56:52.302801 27085 tablet_copy_source_session.cc:205] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.303603 29888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c157f7f72dc4211b256eee0015eed23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.305749 29888 tablet_copy_client.cc:783] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.305931 29888 tablet_copy_client.cc:647] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.307116 29890 tablet_copy_client.cc:1002] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.319124 29903 ts_tablet_manager.cc:902] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1: Initiating tablet copy from peer e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:52.319722 29903 tablet_copy_client.cc:308] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:37115
I20240501 19:56:52.324214 27479 tablet_copy_service.cc:140] P e5057264da53436ea17bad01c66a7fbf: Received BeginTabletCopySession request for tablet 9102f7c8350843c2bd7009ea190860d0 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 ({username='slave'} at 127.20.210.8:39999)
I20240501 19:56:52.324344 27479 tablet_copy_service.cc:161] P e5057264da53436ea17bad01c66a7fbf: Beginning new tablet copy session on tablet 9102f7c8350843c2bd7009ea190860d0 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 at {username='slave'} at 127.20.210.8:39999: session id = 474eb4491dcb47f4a2a2f6e149bc39c1-9102f7c8350843c2bd7009ea190860d0
I20240501 19:56:52.325294 27479 tablet_copy_source_session.cc:205] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.326421 29903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9102f7c8350843c2bd7009ea190860d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.328593 29903 tablet_copy_client.cc:783] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.328827 29903 tablet_copy_client.cc:647] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.330199 29908 tablet_copy_client.cc:1002] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Injecting latency into file download: 1500
W20240501 19:56:52.381433 29842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:56:52.381634 29842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:56:52.381693 29842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:56:52.384644 29842 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:56:52.384698 29842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:56:52.384793 29842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.9
I20240501 19:56:52.388046 29842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.9:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.20.210.9
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.20.210.62:43635
--never_fsync=true
--heap_profile_path=/tmp/kudu.29842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.9
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:56:52.388720 29842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:56:52.389621 29842 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240501 19:56:52.394218 29951 ts_tablet_manager.cc:902] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea: Initiating tablet copy from peer ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:52.394714 29951 tablet_copy_client.cc:308] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:43113
W20240501 19:56:52.397686 29947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:52.398352 29957 ts_tablet_manager.cc:902] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1: Initiating tablet copy from peer ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:52.398345 27217 tablet_copy_service.cc:140] P ef2d33d636574e8fbb9d3b90a15035d6: Received BeginTabletCopySession request for tablet b2c944ca787b475294f9c41f6052a7ea from peer 7ccb4fe66ab942d399551b10f5ed32ea ({username='slave'} at 127.20.210.6:37027)
W20240501 19:56:52.398375 29950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:52.398433 27217 tablet_copy_service.cc:161] P ef2d33d636574e8fbb9d3b90a15035d6: Beginning new tablet copy session on tablet b2c944ca787b475294f9c41f6052a7ea from peer 7ccb4fe66ab942d399551b10f5ed32ea at {username='slave'} at 127.20.210.6:37027: session id = 7ccb4fe66ab942d399551b10f5ed32ea-b2c944ca787b475294f9c41f6052a7ea
I20240501 19:56:52.398885 29957 tablet_copy_client.cc:308] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:43113
W20240501 19:56:52.399014 29948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:56:52.399417 27217 tablet_copy_source_session.cc:205] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.399699 29842 server_base.cc:1034] running on GCE node
I20240501 19:56:52.400044 29842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:56:52.400249 29951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2c944ca787b475294f9c41f6052a7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.400583 29842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:56:52.401751 29842 hybrid_clock.cc:648] HybridClock initialized: now 1714593412401735 us; error 25 us; skew 500 ppm
I20240501 19:56:52.401988 29951 tablet_copy_client.cc:783] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.402182 29951 tablet_copy_client.cc:647] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.402594 27217 tablet_copy_service.cc:140] P ef2d33d636574e8fbb9d3b90a15035d6: Received BeginTabletCopySession request for tablet 896072a285b649dbb31f9f02666ace25 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 ({username='slave'} at 127.20.210.8:33819)
I20240501 19:56:52.402701 27217 tablet_copy_service.cc:161] P ef2d33d636574e8fbb9d3b90a15035d6: Beginning new tablet copy session on tablet 896072a285b649dbb31f9f02666ace25 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 at {username='slave'} at 127.20.210.8:33819: session id = 474eb4491dcb47f4a2a2f6e149bc39c1-896072a285b649dbb31f9f02666ace25
I20240501 19:56:52.403470 27217 tablet_copy_source_session.cc:205] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.403493 29955 tablet_copy_client.cc:1002] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.403849 29842 webserver.cc:460] Webserver started at http://127.20.210.9:40467/ using document root <none> and password file <none>
I20240501 19:56:52.404328 29957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896072a285b649dbb31f9f02666ace25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.404438 29842 fs_manager.cc:359] Metadata directory not provided
I20240501 19:56:52.404506 29842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:56:52.404718 29842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:56:52.406486 29957 tablet_copy_client.cc:783] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.406636 29957 tablet_copy_client.cc:647] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.406647 29842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/instance:
uuid: "045b41d2fb5545fd9975510efab305aa"
format_stamp: "Formatted at 2024-05-01 19:56:52 on dist-test-slave-vgh4"
I20240501 19:56:52.407196 29842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal/instance:
uuid: "045b41d2fb5545fd9975510efab305aa"
format_stamp: "Formatted at 2024-05-01 19:56:52 on dist-test-slave-vgh4"
I20240501 19:56:52.407581 29960 tablet_copy_client.cc:1002] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.408514 29965 ts_tablet_manager.cc:902] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b: Initiating tablet copy from peer e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:52.408977 29965 tablet_copy_client.cc:308] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:37115
I20240501 19:56:52.410671 29842 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 19:56:52.412694 27479 tablet_copy_service.cc:140] P e5057264da53436ea17bad01c66a7fbf: Received BeginTabletCopySession request for tablet e379c346ae524bd28f6e550051c0bf7e from peer 724bae28280242c6982c4f8e315b0d0b ({username='slave'} at 127.20.210.5:41491)
I20240501 19:56:52.412777 27479 tablet_copy_service.cc:161] P e5057264da53436ea17bad01c66a7fbf: Beginning new tablet copy session on tablet e379c346ae524bd28f6e550051c0bf7e from peer 724bae28280242c6982c4f8e315b0d0b at {username='slave'} at 127.20.210.5:41491: session id = 724bae28280242c6982c4f8e315b0d0b-e379c346ae524bd28f6e550051c0bf7e
I20240501 19:56:52.413211 29971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:52.413654 27479 tablet_copy_source_session.cc:205] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.414211 29842 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240501 19:56:52.414335 29842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "045b41d2fb5545fd9975510efab305aa"
format_stamp: "Formatted at 2024-05-01 19:56:52 on dist-test-slave-vgh4"
I20240501 19:56:52.414458 29965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e379c346ae524bd28f6e550051c0bf7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.414412 29842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:56:52.416203 29965 tablet_copy_client.cc:783] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.416366 29965 tablet_copy_client.cc:647] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.417304 29967 tablet_copy_client.cc:1002] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.432078 29842 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:56:52.432543 29842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:56:52.432678 29842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:56:52.433185 29842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:56:52.434072 29842 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:56:52.434116 29842 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:52.434154 29842 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:56:52.434191 29842 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:56:52.442073 29842 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.9:42833
I20240501 19:56:52.442114 30084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.9:42833 every 8 connection(s)
I20240501 19:56:52.442895 29842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240501 19:56:52.447247 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 29842
I20240501 19:56:52.447335 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal/instance
I20240501 19:56:52.450711 30085 heartbeater.cc:347] Connected to a master server at 127.20.210.62:43635
I20240501 19:56:52.450873 30085 heartbeater.cc:464] Registering TS with master...
I20240501 19:56:52.451277 30085 heartbeater.cc:511] Master 127.20.210.62:43635 requested a full tablet report, sending...
I20240501 19:56:52.452091 26916 ts_manager.cc:194] Registered new tserver with Master: 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833)
I20240501 19:56:52.452596 26916 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.9:43431
I20240501 19:56:52.456157 30088 ts_tablet_manager.cc:902] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1: Initiating tablet copy from peer 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:52.456609 30088 tablet_copy_client.cc:308] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:33375
I20240501 19:56:52.459594 27085 tablet_copy_service.cc:140] P 16cfc5a5082742c5b09d35e5c4f7f10e: Received BeginTabletCopySession request for tablet 6aff1d7fd2d2476095ef0a0529110b49 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 ({username='slave'} at 127.20.210.8:42289)
I20240501 19:56:52.459686 27085 tablet_copy_service.cc:161] P 16cfc5a5082742c5b09d35e5c4f7f10e: Beginning new tablet copy session on tablet 6aff1d7fd2d2476095ef0a0529110b49 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 at {username='slave'} at 127.20.210.8:42289: session id = 474eb4491dcb47f4a2a2f6e149bc39c1-6aff1d7fd2d2476095ef0a0529110b49
I20240501 19:56:52.460340 27085 tablet_copy_source_session.cc:205] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.461072 30088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aff1d7fd2d2476095ef0a0529110b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.461988 30093 ts_tablet_manager.cc:902] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:52.462510 30093 tablet_copy_client.cc:308] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:52.462936 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet e3b87a633efe4400931f1e0a566c8f14 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 ({username='slave'} at 127.20.210.7:35965)
I20240501 19:56:52.462998 30088 tablet_copy_client.cc:783] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.463025 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet e3b87a633efe4400931f1e0a566c8f14 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 at {username='slave'} at 127.20.210.7:35965: session id = ecdc0d100f5f4b95b9dda10c2f97cf20-e3b87a633efe4400931f1e0a566c8f14
I20240501 19:56:52.463181 30088 tablet_copy_client.cc:647] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.463814 27348 tablet_copy_source_session.cc:205] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:52.464883 30092 tablet_copy_client.cc:1002] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:52.465409 30093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b87a633efe4400931f1e0a566c8f14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:52.467155 30093 tablet_copy_client.cc:783] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:52.467303 30093 tablet_copy_client.cc:647] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:52.468220 30097 tablet_copy_client.cc:1002] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:53.454139 30085 heartbeater.cc:503] Master 127.20.210.62:43635 was elected leader, sending a full tablet report...
I20240501 19:56:53.514025 29692 tablet_copy_client.cc:523] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.516069 29692 tablet_bootstrap.cc:492] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap starting.
I20240501 19:56:53.521764 29692 log.cc:824] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:53.539402 29723 tablet_copy_client.cc:523] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.541199 29723 tablet_bootstrap.cc:492] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap starting.
I20240501 19:56:53.545511 29692 tablet_bootstrap.cc:492] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.545809 29692 tablet_bootstrap.cc:492] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap complete.
I20240501 19:56:53.546103 29692 ts_tablet_manager.cc:1367] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent bootstrapping tablet: real 0.030s user 0.025s sys 0.002s
I20240501 19:56:53.547096 29723 log.cc:824] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:53.548517 29692 raft_consensus.cc:348] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:53.548761 29692 raft_consensus.cc:729] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7ccb4fe66ab942d399551b10f5ed32ea, State: Initialized, Role: LEARNER
I20240501 19:56:53.549088 29692 consensus_queue.cc:260] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:53.549854 29692 ts_tablet_manager.cc:1398] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20240501 19:56:53.551796 27479 tablet_copy_service.cc:342] P e5057264da53436ea17bad01c66a7fbf: Request end of tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-360f349692784ae69f01ddb9afe73da4 received from {username='slave'} at 127.20.210.6:53755
I20240501 19:56:53.551896 27479 tablet_copy_service.cc:433] P e5057264da53436ea17bad01c66a7fbf: ending tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-360f349692784ae69f01ddb9afe73da4 on tablet 360f349692784ae69f01ddb9afe73da4 with peer 7ccb4fe66ab942d399551b10f5ed32ea
I20240501 19:56:53.570310 29723 tablet_bootstrap.cc:492] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.570650 29723 tablet_bootstrap.cc:492] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap complete.
I20240501 19:56:53.570923 29723 ts_tablet_manager.cc:1367] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent bootstrapping tablet: real 0.030s user 0.028s sys 0.000s
I20240501 19:56:53.573253 29723 raft_consensus.cc:348] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:53.573479 29723 raft_consensus.cc:729] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ecdc0d100f5f4b95b9dda10c2f97cf20, State: Initialized, Role: LEARNER
I20240501 19:56:53.573802 29723 consensus_queue.cc:260] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:53.574599 29723 ts_tablet_manager.cc:1398] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240501 19:56:53.575244 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-6211064bb3ce41f8b19729d8d0de7fe8 received from {username='slave'} at 127.20.210.7:35965
I20240501 19:56:53.575340 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-6211064bb3ce41f8b19729d8d0de7fe8 on tablet 6211064bb3ce41f8b19729d8d0de7fe8 with peer ecdc0d100f5f4b95b9dda10c2f97cf20
I20240501 19:56:53.587117 29754 tablet_copy_client.cc:523] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.588702 29754 tablet_bootstrap.cc:492] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap starting.
I20240501 19:56:53.593847 29754 log.cc:824] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:53.595642 29760 tablet_copy_client.cc:523] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.596673 29760 tablet_bootstrap.cc:492] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap starting.
I20240501 19:56:53.614745 29775 tablet_copy_client.cc:523] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.615857 29775 tablet_bootstrap.cc:492] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap starting.
I20240501 19:56:53.619324 29790 tablet_copy_client.cc:523] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.620412 29790 tablet_bootstrap.cc:492] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap starting.
I20240501 19:56:53.622069 29754 tablet_bootstrap.cc:492] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.622387 29754 tablet_bootstrap.cc:492] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap complete.
I20240501 19:56:53.622655 29754 ts_tablet_manager.cc:1367] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b: Time spent bootstrapping tablet: real 0.034s user 0.029s sys 0.004s
I20240501 19:56:53.625185 29760 tablet_bootstrap.cc:492] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.625479 29760 tablet_bootstrap.cc:492] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap complete.
I20240501 19:56:53.625329 29754 raft_consensus.cc:348] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:53.625573 29754 raft_consensus.cc:729] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 724bae28280242c6982c4f8e315b0d0b, State: Initialized, Role: LEARNER
I20240501 19:56:53.625734 29760 ts_tablet_manager.cc:1367] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent bootstrapping tablet: real 0.029s user 0.025s sys 0.000s
I20240501 19:56:53.625938 29754 consensus_queue.cc:260] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:53.626389 29760 raft_consensus.cc:348] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:53.626554 29760 raft_consensus.cc:729] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ecdc0d100f5f4b95b9dda10c2f97cf20, State: Initialized, Role: LEARNER
I20240501 19:56:53.626688 29760 consensus_queue.cc:260] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:53.626747 29754 ts_tablet_manager.cc:1398] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20240501 19:56:53.627115 29760 ts_tablet_manager.cc:1398] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:53.628484 27085 tablet_copy_service.cc:342] P 16cfc5a5082742c5b09d35e5c4f7f10e: Request end of tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-6c498eacc5e947bfb01dfb90c7039df0 received from {username='slave'} at 127.20.210.7:45013
I20240501 19:56:53.628573 27085 tablet_copy_service.cc:433] P 16cfc5a5082742c5b09d35e5c4f7f10e: ending tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-6c498eacc5e947bfb01dfb90c7039df0 on tablet 6c498eacc5e947bfb01dfb90c7039df0 with peer ecdc0d100f5f4b95b9dda10c2f97cf20
I20240501 19:56:53.629225 27217 tablet_copy_service.cc:342] P ef2d33d636574e8fbb9d3b90a15035d6: Request end of tablet copy session 724bae28280242c6982c4f8e315b0d0b-82f387c0d03a473fb548b447dc867cc7 received from {username='slave'} at 127.20.210.5:41023
I20240501 19:56:53.629323 27217 tablet_copy_service.cc:433] P ef2d33d636574e8fbb9d3b90a15035d6: ending tablet copy session 724bae28280242c6982c4f8e315b0d0b-82f387c0d03a473fb548b447dc867cc7 on tablet 82f387c0d03a473fb548b447dc867cc7 with peer 724bae28280242c6982c4f8e315b0d0b
I20240501 19:56:53.636838 29775 tablet_bootstrap.cc:492] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.637086 29775 tablet_bootstrap.cc:492] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap complete.
I20240501 19:56:53.637240 29775 ts_tablet_manager.cc:1367] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent bootstrapping tablet: real 0.021s user 0.014s sys 0.004s
I20240501 19:56:53.637686 29775 raft_consensus.cc:348] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:53.637801 29775 raft_consensus.cc:729] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7ccb4fe66ab942d399551b10f5ed32ea, State: Initialized, Role: LEARNER
I20240501 19:56:53.637900 29775 consensus_queue.cc:260] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:53.638159 29775 ts_tablet_manager.cc:1398] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:53.638686 27085 tablet_copy_service.cc:342] P 16cfc5a5082742c5b09d35e5c4f7f10e: Request end of tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-199f9529870242f599358995c19250b2 received from {username='slave'} at 127.20.210.6:37711
I20240501 19:56:53.638758 27085 tablet_copy_service.cc:433] P 16cfc5a5082742c5b09d35e5c4f7f10e: ending tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-199f9529870242f599358995c19250b2 on tablet 199f9529870242f599358995c19250b2 with peer 7ccb4fe66ab942d399551b10f5ed32ea
I20240501 19:56:53.642072 29790 tablet_bootstrap.cc:492] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.642320 29790 tablet_bootstrap.cc:492] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap complete.
I20240501 19:56:53.642455 29790 ts_tablet_manager.cc:1367] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent bootstrapping tablet: real 0.022s user 0.019s sys 0.002s
I20240501 19:56:53.642740 29790 raft_consensus.cc:348] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:53.642829 29790 raft_consensus.cc:729] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7ccb4fe66ab942d399551b10f5ed32ea, State: Initialized, Role: LEARNER
I20240501 19:56:53.642912 29790 consensus_queue.cc:260] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:53.643101 29790 ts_tablet_manager.cc:1398] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:53.643478 27085 tablet_copy_service.cc:342] P 16cfc5a5082742c5b09d35e5c4f7f10e: Request end of tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-37288cafed764ba18fcc07b6e3c7a722 received from {username='slave'} at 127.20.210.6:37711
I20240501 19:56:53.643541 27085 tablet_copy_service.cc:433] P 16cfc5a5082742c5b09d35e5c4f7f10e: ending tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-37288cafed764ba18fcc07b6e3c7a722 on tablet 37288cafed764ba18fcc07b6e3c7a722 with peer 7ccb4fe66ab942d399551b10f5ed32ea
I20240501 19:56:53.645129 29795 tablet_copy_client.cc:523] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.646291 29795 tablet_bootstrap.cc:492] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap starting.
I20240501 19:56:53.661846 29795 tablet_bootstrap.cc:492] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.662055 29795 tablet_bootstrap.cc:492] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap complete.
I20240501 19:56:53.662171 29795 ts_tablet_manager.cc:1367] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20240501 19:56:53.662500 29795 raft_consensus.cc:348] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:53.662588 29795 raft_consensus.cc:729] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ecdc0d100f5f4b95b9dda10c2f97cf20, State: Initialized, Role: LEARNER
I20240501 19:56:53.662675 29795 consensus_queue.cc:260] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:53.662890 29795 ts_tablet_manager.cc:1398] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent starting tablet: real 0.001s user 0.000s sys 0.003s
I20240501 19:56:53.663293 27217 tablet_copy_service.cc:342] P ef2d33d636574e8fbb9d3b90a15035d6: Request end of tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-a74a8248e5584fb483728f7676502ad5 received from {username='slave'} at 127.20.210.7:46547
I20240501 19:56:53.663372 27217 tablet_copy_service.cc:433] P ef2d33d636574e8fbb9d3b90a15035d6: ending tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-a74a8248e5584fb483728f7676502ad5 on tablet a74a8248e5584fb483728f7676502ad5 with peer ecdc0d100f5f4b95b9dda10c2f97cf20
I20240501 19:56:53.679184 29008 raft_consensus.cc:1203] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:53.683815 28746 raft_consensus.cc:1203] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:53.694612 29811 tablet_copy_client.cc:523] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.695732 29811 tablet_bootstrap.cc:492] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap starting.
I20240501 19:56:53.711566 29811 tablet_bootstrap.cc:492] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.711781 29811 tablet_bootstrap.cc:492] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap complete.
I20240501 19:56:53.711930 29811 ts_tablet_manager.cc:1367] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b: Time spent bootstrapping tablet: real 0.016s user 0.015s sys 0.000s
I20240501 19:56:53.712320 29811 raft_consensus.cc:348] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:53.712414 29811 raft_consensus.cc:729] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 724bae28280242c6982c4f8e315b0d0b, State: Initialized, Role: LEARNER
I20240501 19:56:53.712508 29811 consensus_queue.cc:260] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:53.712697 29811 ts_tablet_manager.cc:1398] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:53.713161 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session 724bae28280242c6982c4f8e315b0d0b-ae366b11dc9f492b9467d68947cd3dc1 received from {username='slave'} at 127.20.210.5:40583
I20240501 19:56:53.713243 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session 724bae28280242c6982c4f8e315b0d0b-ae366b11dc9f492b9467d68947cd3dc1 on tablet ae366b11dc9f492b9467d68947cd3dc1 with peer 724bae28280242c6982c4f8e315b0d0b
I20240501 19:56:53.722272 29827 tablet_copy_client.cc:523] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.723256 29827 tablet_bootstrap.cc:492] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b: Bootstrap starting.
I20240501 19:56:53.730461 29816 tablet_copy_client.cc:523] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.730461 29817 tablet_copy_client.cc:523] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.732347 29817 tablet_bootstrap.cc:492] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap starting.
I20240501 19:56:53.732331 29816 tablet_bootstrap.cc:492] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap starting.
I20240501 19:56:53.738445 29816 log.cc:824] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:53.738445 29817 log.cc:824] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1: Log is configured to *not* fsync() on all Append() calls
I20240501 19:56:53.739352 29827 tablet_bootstrap.cc:492] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.739555 29827 tablet_bootstrap.cc:492] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b: Bootstrap complete.
I20240501 19:56:53.739799 29827 ts_tablet_manager.cc:1367] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b: Time spent bootstrapping tablet: real 0.017s user 0.015s sys 0.003s
I20240501 19:56:53.740118 29827 raft_consensus.cc:348] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:53.740211 29827 raft_consensus.cc:729] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 724bae28280242c6982c4f8e315b0d0b, State: Initialized, Role: LEARNER
I20240501 19:56:53.740304 29827 consensus_queue.cc:260] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:53.740901 29827 ts_tablet_manager.cc:1398] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:53.741268 27217 tablet_copy_service.cc:342] P ef2d33d636574e8fbb9d3b90a15035d6: Request end of tablet copy session 724bae28280242c6982c4f8e315b0d0b-0f417f4001bc48158187ed262f7737ab received from {username='slave'} at 127.20.210.5:41023
I20240501 19:56:53.741364 27217 tablet_copy_service.cc:433] P ef2d33d636574e8fbb9d3b90a15035d6: ending tablet copy session 724bae28280242c6982c4f8e315b0d0b-0f417f4001bc48158187ed262f7737ab on tablet 0f417f4001bc48158187ed262f7737ab with peer 724bae28280242c6982c4f8e315b0d0b
I20240501 19:56:53.756590 29817 tablet_bootstrap.cc:492] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.756868 29816 tablet_bootstrap.cc:492] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.756883 29817 tablet_bootstrap.cc:492] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap complete.
I20240501 19:56:53.757135 29816 tablet_bootstrap.cc:492] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap complete.
I20240501 19:56:53.757165 29817 ts_tablet_manager.cc:1367] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent bootstrapping tablet: real 0.025s user 0.022s sys 0.001s
I20240501 19:56:53.757287 29816 ts_tablet_manager.cc:1367] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent bootstrapping tablet: real 0.025s user 0.021s sys 0.000s
I20240501 19:56:53.758272 29843 tablet_copy_client.cc:523] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.759012 29817 raft_consensus.cc:348] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:53.759014 29816 raft_consensus.cc:348] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:53.759255 29817 raft_consensus.cc:729] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 474eb4491dcb47f4a2a2f6e149bc39c1, State: Initialized, Role: LEARNER
I20240501 19:56:53.759256 29816 raft_consensus.cc:729] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 474eb4491dcb47f4a2a2f6e149bc39c1, State: Initialized, Role: LEARNER
I20240501 19:56:53.759553 29817 consensus_queue.cc:260] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:53.759617 29843 tablet_bootstrap.cc:492] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap starting.
I20240501 19:56:53.759555 29816 consensus_queue.cc:260] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:53.760299 29816 ts_tablet_manager.cc:1398] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20240501 19:56:53.760299 29817 ts_tablet_manager.cc:1398] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240501 19:56:53.761067 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-5ecba34e358c41d9b812113c955b6b29 received from {username='slave'} at 127.20.210.8:56643
I20240501 19:56:53.761169 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-5ecba34e358c41d9b812113c955b6b29 on tablet 5ecba34e358c41d9b812113c955b6b29 with peer 474eb4491dcb47f4a2a2f6e149bc39c1
I20240501 19:56:53.761461 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-3c80c1dcbcd3439ba75066259557e716 received from {username='slave'} at 127.20.210.8:56643
I20240501 19:56:53.761540 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-3c80c1dcbcd3439ba75066259557e716 on tablet 3c80c1dcbcd3439ba75066259557e716 with peer 474eb4491dcb47f4a2a2f6e149bc39c1
I20240501 19:56:53.765658 28877 raft_consensus.cc:1203] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:53.777069 29843 tablet_bootstrap.cc:492] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.777271 29843 tablet_bootstrap.cc:492] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap complete.
I20240501 19:56:53.777405 29843 ts_tablet_manager.cc:1367] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.005s
I20240501 19:56:53.777702 29843 raft_consensus.cc:348] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:53.777789 29843 raft_consensus.cc:729] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ecdc0d100f5f4b95b9dda10c2f97cf20, State: Initialized, Role: LEARNER
I20240501 19:56:53.777864 29843 consensus_queue.cc:260] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:53.778079 29843 ts_tablet_manager.cc:1398] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:53.778482 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-f149fd8790a0412e81ac49d0ce359ede received from {username='slave'} at 127.20.210.7:35965
I20240501 19:56:53.778564 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-f149fd8790a0412e81ac49d0ce359ede on tablet f149fd8790a0412e81ac49d0ce359ede with peer ecdc0d100f5f4b95b9dda10c2f97cf20
I20240501 19:56:53.807402 29888 tablet_copy_client.cc:523] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.807610 29008 raft_consensus.cc:1203] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:53.808805 29888 tablet_bootstrap.cc:492] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap starting.
I20240501 19:56:53.826823 28877 raft_consensus.cc:1203] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:53.829403 29888 tablet_bootstrap.cc:492] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.829733 29888 tablet_bootstrap.cc:492] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap complete.
I20240501 19:56:53.829908 29888 ts_tablet_manager.cc:1367] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent bootstrapping tablet: real 0.021s user 0.021s sys 0.000s
I20240501 19:56:53.830461 29903 tablet_copy_client.cc:523] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.830309 29888 raft_consensus.cc:348] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:53.830549 29888 raft_consensus.cc:729] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7ccb4fe66ab942d399551b10f5ed32ea, State: Initialized, Role: LEARNER
I20240501 19:56:53.830649 29888 consensus_queue.cc:260] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:53.830946 29888 ts_tablet_manager.cc:1398] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:53.831688 27085 tablet_copy_service.cc:342] P 16cfc5a5082742c5b09d35e5c4f7f10e: Request end of tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-8c157f7f72dc4211b256eee0015eed23 received from {username='slave'} at 127.20.210.6:37711
I20240501 19:56:53.831750 27085 tablet_copy_service.cc:433] P 16cfc5a5082742c5b09d35e5c4f7f10e: ending tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-8c157f7f72dc4211b256eee0015eed23 on tablet 8c157f7f72dc4211b256eee0015eed23 with peer 7ccb4fe66ab942d399551b10f5ed32ea
I20240501 19:56:53.831840 29903 tablet_bootstrap.cc:492] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap starting.
I20240501 19:56:53.853698 29903 tablet_bootstrap.cc:492] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.853931 29903 tablet_bootstrap.cc:492] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap complete.
I20240501 19:56:53.854084 29903 ts_tablet_manager.cc:1367] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent bootstrapping tablet: real 0.022s user 0.019s sys 0.000s
I20240501 19:56:53.854465 29903 raft_consensus.cc:348] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:53.854560 29903 raft_consensus.cc:729] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 474eb4491dcb47f4a2a2f6e149bc39c1, State: Initialized, Role: LEARNER
I20240501 19:56:53.854638 29903 consensus_queue.cc:260] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:53.854910 29903 ts_tablet_manager.cc:1398] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 19:56:53.855396 27479 tablet_copy_service.cc:342] P e5057264da53436ea17bad01c66a7fbf: Request end of tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-9102f7c8350843c2bd7009ea190860d0 received from {username='slave'} at 127.20.210.8:39999
I20240501 19:56:53.855463 27479 tablet_copy_service.cc:433] P e5057264da53436ea17bad01c66a7fbf: ending tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-9102f7c8350843c2bd7009ea190860d0 on tablet 9102f7c8350843c2bd7009ea190860d0 with peer 474eb4491dcb47f4a2a2f6e149bc39c1
I20240501 19:56:53.895823 28877 raft_consensus.cc:1203] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:53.900151 28877 raft_consensus.cc:1203] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:53.903750 29951 tablet_copy_client.cc:523] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.905434 29951 tablet_bootstrap.cc:492] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap starting.
I20240501 19:56:53.907873 29957 tablet_copy_client.cc:523] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.909301 29957 tablet_bootstrap.cc:492] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap starting.
I20240501 19:56:53.909924 29008 raft_consensus.cc:1203] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:53.917583 29965 tablet_copy_client.cc:523] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.918788 29965 tablet_bootstrap.cc:492] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b: Bootstrap starting.
I20240501 19:56:53.928970 29951 tablet_bootstrap.cc:492] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.929229 29951 tablet_bootstrap.cc:492] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap complete.
I20240501 19:56:53.929383 29951 ts_tablet_manager.cc:1367] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent bootstrapping tablet: real 0.024s user 0.017s sys 0.005s
I20240501 19:56:53.929838 29951 raft_consensus.cc:348] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:53.929960 29951 raft_consensus.cc:729] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7ccb4fe66ab942d399551b10f5ed32ea, State: Initialized, Role: LEARNER
I20240501 19:56:53.930053 29951 consensus_queue.cc:260] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:53.930876 29951 ts_tablet_manager.cc:1398] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent starting tablet: real 0.001s user 0.003s sys 0.001s
I20240501 19:56:53.931707 27217 tablet_copy_service.cc:342] P ef2d33d636574e8fbb9d3b90a15035d6: Request end of tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-b2c944ca787b475294f9c41f6052a7ea received from {username='slave'} at 127.20.210.6:37027
I20240501 19:56:53.931813 27217 tablet_copy_service.cc:433] P ef2d33d636574e8fbb9d3b90a15035d6: ending tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-b2c944ca787b475294f9c41f6052a7ea on tablet b2c944ca787b475294f9c41f6052a7ea with peer 7ccb4fe66ab942d399551b10f5ed32ea
I20240501 19:56:53.938060 29957 tablet_bootstrap.cc:492] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.938359 29957 tablet_bootstrap.cc:492] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap complete.
I20240501 19:56:53.938508 29957 ts_tablet_manager.cc:1367] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent bootstrapping tablet: real 0.029s user 0.024s sys 0.003s
I20240501 19:56:53.938850 29957 raft_consensus.cc:348] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:53.938946 29957 raft_consensus.cc:729] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 474eb4491dcb47f4a2a2f6e149bc39c1, State: Initialized, Role: LEARNER
I20240501 19:56:53.939036 29957 consensus_queue.cc:260] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:53.939325 29957 ts_tablet_manager.cc:1398] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:53.939839 27217 tablet_copy_service.cc:342] P ef2d33d636574e8fbb9d3b90a15035d6: Request end of tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-896072a285b649dbb31f9f02666ace25 received from {username='slave'} at 127.20.210.8:33819
I20240501 19:56:53.939901 27217 tablet_copy_service.cc:433] P ef2d33d636574e8fbb9d3b90a15035d6: ending tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-896072a285b649dbb31f9f02666ace25 on tablet 896072a285b649dbb31f9f02666ace25 with peer 474eb4491dcb47f4a2a2f6e149bc39c1
I20240501 19:56:53.941939 29965 tablet_bootstrap.cc:492] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.942137 29965 tablet_bootstrap.cc:492] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b: Bootstrap complete.
I20240501 19:56:53.942277 29965 ts_tablet_manager.cc:1367] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b: Time spent bootstrapping tablet: real 0.024s user 0.022s sys 0.000s
I20240501 19:56:53.942567 29965 raft_consensus.cc:348] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:53.942653 29965 raft_consensus.cc:729] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 724bae28280242c6982c4f8e315b0d0b, State: Initialized, Role: LEARNER
I20240501 19:56:53.942726 29965 consensus_queue.cc:260] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:53.942937 29965 ts_tablet_manager.cc:1398] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:53.943420 27479 tablet_copy_service.cc:342] P e5057264da53436ea17bad01c66a7fbf: Request end of tablet copy session 724bae28280242c6982c4f8e315b0d0b-e379c346ae524bd28f6e550051c0bf7e received from {username='slave'} at 127.20.210.5:41491
I20240501 19:56:53.943491 27479 tablet_copy_service.cc:433] P e5057264da53436ea17bad01c66a7fbf: ending tablet copy session 724bae28280242c6982c4f8e315b0d0b-e379c346ae524bd28f6e550051c0bf7e on tablet e379c346ae524bd28f6e550051c0bf7e with peer 724bae28280242c6982c4f8e315b0d0b
I20240501 19:56:53.945627 28746 raft_consensus.cc:1203] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:53.965132 30088 tablet_copy_client.cc:523] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.966169 30088 tablet_bootstrap.cc:492] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap starting.
I20240501 19:56:53.968480 30093 tablet_copy_client.cc:523] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:53.969491 30093 tablet_bootstrap.cc:492] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap starting.
I20240501 19:56:53.983868 30088 tablet_bootstrap.cc:492] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.984122 30088 tablet_bootstrap.cc:492] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap complete.
I20240501 19:56:53.984263 30088 ts_tablet_manager.cc:1367] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent bootstrapping tablet: real 0.018s user 0.017s sys 0.000s
I20240501 19:56:53.984620 30088 raft_consensus.cc:348] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:53.984730 30088 raft_consensus.cc:729] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 474eb4491dcb47f4a2a2f6e149bc39c1, State: Initialized, Role: LEARNER
I20240501 19:56:53.984819 30088 consensus_queue.cc:260] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:53.985077 30088 ts_tablet_manager.cc:1398] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:53.985463 27085 tablet_copy_service.cc:342] P 16cfc5a5082742c5b09d35e5c4f7f10e: Request end of tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-6aff1d7fd2d2476095ef0a0529110b49 received from {username='slave'} at 127.20.210.8:42289
I20240501 19:56:53.985538 27085 tablet_copy_service.cc:433] P 16cfc5a5082742c5b09d35e5c4f7f10e: ending tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-6aff1d7fd2d2476095ef0a0529110b49 on tablet 6aff1d7fd2d2476095ef0a0529110b49 with peer 474eb4491dcb47f4a2a2f6e149bc39c1
I20240501 19:56:53.989802 30093 tablet_bootstrap.cc:492] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:53.990007 30093 tablet_bootstrap.cc:492] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap complete.
I20240501 19:56:53.990165 30093 ts_tablet_manager.cc:1367] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent bootstrapping tablet: real 0.021s user 0.013s sys 0.004s
I20240501 19:56:53.990528 30093 raft_consensus.cc:348] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:53.990628 30093 raft_consensus.cc:729] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ecdc0d100f5f4b95b9dda10c2f97cf20, State: Initialized, Role: LEARNER
I20240501 19:56:53.990720 30093 consensus_queue.cc:260] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:53.990942 30093 ts_tablet_manager.cc:1398] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 19:56:53.991366 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-e3b87a633efe4400931f1e0a566c8f14 received from {username='slave'} at 127.20.210.7:35965
I20240501 19:56:53.991439 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-e3b87a633efe4400931f1e0a566c8f14 on tablet e3b87a633efe4400931f1e0a566c8f14 with peer ecdc0d100f5f4b95b9dda10c2f97cf20
I20240501 19:56:54.065749 29140 raft_consensus.cc:1203] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:54.100453 29008 raft_consensus.cc:1203] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:54.195881 30654 raft_consensus.cc:1050] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6: attempting to promote NON_VOTER 724bae28280242c6982c4f8e315b0d0b to VOTER
I20240501 19:56:54.196462 30654 consensus_queue.cc:237] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:56:54.197847 27457 raft_consensus.cc:1261] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.197997 28746 raft_consensus.cc:1261] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.198161 27066 raft_consensus.cc:1261] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.198410 30654 consensus_queue.cc:1035] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.198578 30173 consensus_queue.cc:1035] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.198851 30657 consensus_queue.cc:1035] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.200433 30173 raft_consensus.cc:1050] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6: attempting to promote NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 to VOTER
I20240501 19:56:54.200745 30173 consensus_queue.cc:237] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:54.201161 30654 raft_consensus.cc:2932] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.201503 27457 raft_consensus.cc:2932] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.202178 29008 raft_consensus.cc:1261] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.202419 27326 raft_consensus.cc:1261] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.203425 27066 raft_consensus.cc:1261] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.203521 27064 raft_consensus.cc:2932] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.204236 30657 consensus_queue.cc:1035] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.204370 26920 catalog_manager.cc:5484] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.206359 30173 consensus_queue.cc:1035] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.206535 30654 consensus_queue.cc:1035] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.207043 28746 raft_consensus.cc:2932] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.210873 30173 raft_consensus.cc:2932] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.211705 27066 raft_consensus.cc:2932] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.212504 27326 raft_consensus.cc:2932] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.213356 29008 raft_consensus.cc:2932] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.215462 27197 consensus_queue.cc:237] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:56:54.216553 28746 raft_consensus.cc:1261] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.217072 30655 consensus_queue.cc:1035] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.217192 26923 catalog_manager.cc:5484] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.217469 27066 raft_consensus.cc:1261] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.217764 30655 consensus_queue.cc:1035] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.218842 30655 raft_consensus.cc:2932] T 82f387c0d03a473fb548b447dc867cc7 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.219645 27066 raft_consensus.cc:2932] T 82f387c0d03a473fb548b447dc867cc7 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.220158 26908 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 82f387c0d03a473fb548b447dc867cc7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.221094 28746 raft_consensus.cc:2932] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
W20240501 19:56:54.223134 26908 catalog_manager.cc:5007] ChangeConfig:REMOVE_PEER RPC for tablet 82f387c0d03a473fb548b447dc867cc7 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 5 but the committed config has opid_index of 6
I20240501 19:56:54.224438 26924 catalog_manager.cc:5484] T 82f387c0d03a473fb548b447dc867cc7 P 724bae28280242c6982c4f8e315b0d0b reported cstate change: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.224531 27197 consensus_queue.cc:237] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:54.225841 28746 raft_consensus.cc:1203] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:54.226301 27066 raft_consensus.cc:1261] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.226368 29008 raft_consensus.cc:1261] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.226770 30657 consensus_queue.cc:1035] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.226923 30654 consensus_queue.cc:1035] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.228106 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet 82f387c0d03a473fb548b447dc867cc7 with delete_type TABLET_DATA_TOMBSTONED (TS e5057264da53436ea17bad01c66a7fbf not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41962
I20240501 19:56:54.228564 30657 raft_consensus.cc:2932] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.229246 30099 raft_consensus.cc:1050] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf: attempting to promote NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea to VOTER
I20240501 19:56:54.229699 30099 consensus_queue.cc:237] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:54.230448 26908 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a74a8248e5584fb483728f7676502ad5 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.230697 27066 raft_consensus.cc:2932] T a74a8248e5584fb483728f7676502ad5 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.230863 29008 raft_consensus.cc:2932] T a74a8248e5584fb483728f7676502ad5 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.231621 27197 raft_consensus.cc:1261] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.232137 30099 consensus_queue.cc:1035] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.232424 26920 catalog_manager.cc:5484] T a74a8248e5584fb483728f7676502ad5 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.232697 27326 raft_consensus.cc:1261] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.234867 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet a74a8248e5584fb483728f7676502ad5 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3f70fdad7b438a841f74fc638d17f5 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34348
I20240501 19:56:54.235486 30099 consensus_queue.cc:1035] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.236634 28877 raft_consensus.cc:1261] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.237066 30099 consensus_queue.cc:1035] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.237784 30686 tablet_replica.cc:330] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf: stopping tablet replica
I20240501 19:56:54.238075 30688 raft_consensus.cc:2932] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.238425 30686 raft_consensus.cc:2229] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.238656 30686 raft_consensus.cc:2258] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.239032 30691 tablet_replica.cc:330] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5: stopping tablet replica
I20240501 19:56:54.239580 30691 raft_consensus.cc:2229] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.240378 27326 raft_consensus.cc:2932] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.240643 30691 raft_consensus.cc:2258] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.240614 27197 raft_consensus.cc:2932] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.240810 26915 catalog_manager.cc:5484] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.243072 30691 ts_tablet_manager.cc:1875] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.243992 28877 raft_consensus.cc:2932] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.244637 30686 ts_tablet_manager.cc:1875] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.245854 30691 ts_tablet_manager.cc:1888] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.246515 30691 log.cc:1196] T a74a8248e5584fb483728f7676502ad5 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/a74a8248e5584fb483728f7676502ad5
I20240501 19:56:54.247104 26910 catalog_manager.cc:4830] TS 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155): tablet a74a8248e5584fb483728f7676502ad5 (table rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]) successfully deleted
I20240501 19:56:54.248440 27457 consensus_queue.cc:237] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:54.248723 30686 ts_tablet_manager.cc:1888] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.248807 30686 log.cc:1196] T 82f387c0d03a473fb548b447dc867cc7 P e5057264da53436ea17bad01c66a7fbf: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/82f387c0d03a473fb548b447dc867cc7
I20240501 19:56:54.249445 26911 catalog_manager.cc:4830] TS e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115): tablet 82f387c0d03a473fb548b447dc867cc7 (table rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]) successfully deleted
I20240501 19:56:54.249524 28877 raft_consensus.cc:1261] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.249666 27326 raft_consensus.cc:1261] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.249881 30099 consensus_queue.cc:1035] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.250166 30099 consensus_queue.cc:1035] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.251508 30099 raft_consensus.cc:2932] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.252265 27326 raft_consensus.cc:2932] T 360f349692784ae69f01ddb9afe73da4 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.252779 26911 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 360f349692784ae69f01ddb9afe73da4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.253316 28876 raft_consensus.cc:2932] T 360f349692784ae69f01ddb9afe73da4 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.254091 26915 catalog_manager.cc:5484] T 360f349692784ae69f01ddb9afe73da4 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.256740 27175 tablet_service.cc:1514] Processing DeleteTablet for tablet 360f349692784ae69f01ddb9afe73da4 with delete_type TABLET_DATA_TOMBSTONED (TS ef2d33d636574e8fbb9d3b90a15035d6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43438
I20240501 19:56:54.257197 30704 tablet_replica.cc:330] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6: stopping tablet replica
I20240501 19:56:54.257504 30704 raft_consensus.cc:2229] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.257692 30704 raft_consensus.cc:2258] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.258600 30704 ts_tablet_manager.cc:1875] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.260780 30704 ts_tablet_manager.cc:1888] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.260869 30704 log.cc:1196] T 360f349692784ae69f01ddb9afe73da4 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/360f349692784ae69f01ddb9afe73da4
I20240501 19:56:54.261432 26908 catalog_manager.cc:4830] TS ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113): tablet 360f349692784ae69f01ddb9afe73da4 (table rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]) successfully deleted
I20240501 19:56:54.263540 29140 raft_consensus.cc:1203] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:54.275679 28876 raft_consensus.cc:1203] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:54.279094 29140 raft_consensus.cc:1203] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:54.288087 29140 raft_consensus.cc:1203] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:54.288148 30725 raft_consensus.cc:1050] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e: attempting to promote NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea to VOTER
I20240501 19:56:54.288620 30725 consensus_queue.cc:237] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:54.289829 27457 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.289980 27197 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.290205 28876 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.290306 29484 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.290714 30727 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.290869 29484 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.292200 29484 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.292605 27457 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.293022 28876 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.294060 27197 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.294713 26923 catalog_manager.cc:5484] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.297986 27066 consensus_queue.cc:237] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:54.298871 28876 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.299245 30725 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.299391 27457 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.299623 30603 raft_consensus.cc:1050] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e: attempting to promote NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea to VOTER
I20240501 19:56:54.299718 29484 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.299875 30603 consensus_queue.cc:237] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:54.301016 27197 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.301031 28877 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.301293 27459 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.301316 30727 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.301637 30727 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.301932 30727 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.302027 29484 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.302429 28877 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.303678 26909 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 37288cafed764ba18fcc07b6e3c7a722 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.304541 26924 catalog_manager.cc:5484] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea reported cstate change: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.305552 30603 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.305738 27457 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.306402 27459 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.307348 27197 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.308535 27175 tablet_service.cc:1514] Processing DeleteTablet for tablet 37288cafed764ba18fcc07b6e3c7a722 with delete_type TABLET_DATA_TOMBSTONED (TS ef2d33d636574e8fbb9d3b90a15035d6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43438
I20240501 19:56:54.308454 28877 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.308713 30704 tablet_replica.cc:330] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6: stopping tablet replica
I20240501 19:56:54.308831 30704 raft_consensus.cc:2229] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.308972 30704 raft_consensus.cc:2258] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.310066 30704 ts_tablet_manager.cc:1875] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.310156 26924 catalog_manager.cc:5484] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.311965 30704 ts_tablet_manager.cc:1888] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.312038 30704 log.cc:1196] T 37288cafed764ba18fcc07b6e3c7a722 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/37288cafed764ba18fcc07b6e3c7a722
I20240501 19:56:54.312582 29008 raft_consensus.cc:1203] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:54.318584 26908 catalog_manager.cc:4830] TS ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113): tablet 37288cafed764ba18fcc07b6e3c7a722 (table rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]) successfully deleted
I20240501 19:56:54.318779 27066 consensus_queue.cc:237] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:54.319689 28877 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.319844 27457 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.319974 29484 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.320174 30603 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.321480 28746 raft_consensus.cc:1203] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:54.321429 30603 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.321806 27459 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.322659 26909 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 199f9529870242f599358995c19250b2 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.323194 28877 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.323990 26923 catalog_manager.cc:5484] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.326128 27175 tablet_service.cc:1514] Processing DeleteTablet for tablet 199f9529870242f599358995c19250b2 with delete_type TABLET_DATA_TOMBSTONED (TS ef2d33d636574e8fbb9d3b90a15035d6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43438
I20240501 19:56:54.326352 30704 tablet_replica.cc:330] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6: stopping tablet replica
I20240501 19:56:54.326484 30704 raft_consensus.cc:2229] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.326609 30704 raft_consensus.cc:2258] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.327137 30704 ts_tablet_manager.cc:1875] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.328437 30704 ts_tablet_manager.cc:1888] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.328492 30704 log.cc:1196] T 199f9529870242f599358995c19250b2 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/199f9529870242f599358995c19250b2
I20240501 19:56:54.328871 26908 catalog_manager.cc:4830] TS ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113): tablet 199f9529870242f599358995c19250b2 (table rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]) successfully deleted
I20240501 19:56:54.399577 30735 raft_consensus.cc:1050] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 to VOTER
I20240501 19:56:54.400004 30735 consensus_queue.cc:237] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:54.401065 27066 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.401088 27457 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.401109 29008 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.401429 30121 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.401595 30735 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.401757 29433 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.403637 29433 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.404042 27066 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.405676 27457 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.405719 29008 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.405946 26915 catalog_manager.cc:5484] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.408905 27326 consensus_queue.cc:237] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:54.409797 29008 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.409843 27066 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.410096 30737 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.410279 30735 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.411581 30121 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.412025 29008 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.412078 27066 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.413792 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f149fd8790a0412e81ac49d0ce359ede with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.413832 26923 catalog_manager.cc:5484] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 reported cstate change: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.415509 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet f149fd8790a0412e81ac49d0ce359ede with delete_type TABLET_DATA_TOMBSTONED (TS e5057264da53436ea17bad01c66a7fbf not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41962
I20240501 19:56:54.415956 30686 tablet_replica.cc:330] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf: stopping tablet replica
I20240501 19:56:54.416101 30686 raft_consensus.cc:2229] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.416236 30686 raft_consensus.cc:2258] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.416810 30686 ts_tablet_manager.cc:1875] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.418391 30686 ts_tablet_manager.cc:1888] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.418483 30686 log.cc:1196] T f149fd8790a0412e81ac49d0ce359ede P e5057264da53436ea17bad01c66a7fbf: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/f149fd8790a0412e81ac49d0ce359ede
I20240501 19:56:54.418931 26911 catalog_manager.cc:4830] TS e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115): tablet f149fd8790a0412e81ac49d0ce359ede (table rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]) successfully deleted
I20240501 19:56:54.434748 29433 raft_consensus.cc:1050] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER 724bae28280242c6982c4f8e315b0d0b to VOTER
I20240501 19:56:54.435101 29433 consensus_queue.cc:237] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:56:54.436316 27457 raft_consensus.cc:1261] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.436342 27197 raft_consensus.cc:1261] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.436388 28746 raft_consensus.cc:1261] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.436709 30737 consensus_queue.cc:1035] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.436885 29433 consensus_queue.cc:1035] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.437052 30735 consensus_queue.cc:1035] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.438746 29433 raft_consensus.cc:2932] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.439075 27197 raft_consensus.cc:2932] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.440546 28746 raft_consensus.cc:2932] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.441031 26920 catalog_manager.cc:5484] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.441645 27457 raft_consensus.cc:2932] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.446910 27326 consensus_queue.cc:237] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:56:54.447835 30121 raft_consensus.cc:1050] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 to VOTER
I20240501 19:56:54.448033 28746 raft_consensus.cc:1261] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.448134 30121 consensus_queue.cc:237] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:54.448321 30737 consensus_queue.cc:1035] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.449323 29008 raft_consensus.cc:1261] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.449471 27066 raft_consensus.cc:1261] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.449471 27197 raft_consensus.cc:1261] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.449705 27457 raft_consensus.cc:1261] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.449858 30737 consensus_queue.cc:1035] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.450017 29433 consensus_queue.cc:1035] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.450325 29433 consensus_queue.cc:1035] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.450475 29140 raft_consensus.cc:1203] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:54.451057 29433 raft_consensus.cc:2932] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.452220 27457 raft_consensus.cc:2932] T ae366b11dc9f492b9467d68947cd3dc1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.452344 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ae366b11dc9f492b9467d68947cd3dc1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.452734 30121 consensus_queue.cc:1035] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.453642 30737 raft_consensus.cc:2932] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.454115 28746 raft_consensus.cc:2932] T ae366b11dc9f492b9467d68947cd3dc1 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.454557 26923 catalog_manager.cc:5484] T ae366b11dc9f492b9467d68947cd3dc1 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.455974 27197 raft_consensus.cc:2932] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.456334 27066 raft_consensus.cc:2932] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.456871 27175 tablet_service.cc:1514] Processing DeleteTablet for tablet ae366b11dc9f492b9467d68947cd3dc1 with delete_type TABLET_DATA_TOMBSTONED (TS ef2d33d636574e8fbb9d3b90a15035d6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43438
I20240501 19:56:54.457065 30704 tablet_replica.cc:330] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6: stopping tablet replica
I20240501 19:56:54.457182 30704 raft_consensus.cc:2229] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.457324 30704 raft_consensus.cc:2258] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.458333 30704 ts_tablet_manager.cc:1875] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.458999 26920 catalog_manager.cc:5484] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.459717 29008 raft_consensus.cc:2932] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.460247 30704 ts_tablet_manager.cc:1888] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.460304 30704 log.cc:1196] T ae366b11dc9f492b9467d68947cd3dc1 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/ae366b11dc9f492b9467d68947cd3dc1
I20240501 19:56:54.460808 26908 catalog_manager.cc:4830] TS ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113): tablet ae366b11dc9f492b9467d68947cd3dc1 (table rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]) successfully deleted
I20240501 19:56:54.500746 30725 raft_consensus.cc:1050] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e: attempting to promote NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea to VOTER
I20240501 19:56:54.501062 30725 consensus_queue.cc:237] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:54.503234 27326 raft_consensus.cc:1261] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.503283 27197 raft_consensus.cc:1261] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.503837 28877 raft_consensus.cc:1261] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.504285 30603 consensus_queue.cc:1035] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.504979 29484 consensus_queue.cc:1035] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.505275 29484 consensus_queue.cc:1035] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.507356 29484 raft_consensus.cc:2932] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.508427 28877 raft_consensus.cc:2932] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.508451 27197 raft_consensus.cc:2932] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.510783 26920 catalog_manager.cc:5484] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea reported cstate change: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.510957 27327 raft_consensus.cc:2932] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.513509 27066 consensus_queue.cc:237] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:54.514739 27197 raft_consensus.cc:1261] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.515143 29484 consensus_queue.cc:1035] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.515621 29484 raft_consensus.cc:1050] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: attempting to promote NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 to VOTER
I20240501 19:56:54.515980 29484 consensus_queue.cc:237] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:54.516414 28877 raft_consensus.cc:1261] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.516990 29484 consensus_queue.cc:1035] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.517403 27457 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.517764 27196 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.518026 29008 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.518376 30727 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.518561 30725 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.518950 29484 raft_consensus.cc:2932] T 8c157f7f72dc4211b256eee0015eed23 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.519258 27197 raft_consensus.cc:2932] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.519973 30603 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.520788 26909 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8c157f7f72dc4211b256eee0015eed23 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.521375 26923 catalog_manager.cc:5484] T 8c157f7f72dc4211b256eee0015eed23 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.522538 30725 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.524205 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c157f7f72dc4211b256eee0015eed23 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3f70fdad7b438a841f74fc638d17f5 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34348
I20240501 19:56:54.524209 27457 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.524420 30691 tablet_replica.cc:330] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5: stopping tablet replica
I20240501 19:56:54.524546 30691 raft_consensus.cc:2229] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.524694 30691 raft_consensus.cc:2258] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.524798 27197 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.525584 30691 ts_tablet_manager.cc:1875] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.526355 29008 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.527283 28877 raft_consensus.cc:2932] T 8c157f7f72dc4211b256eee0015eed23 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.527597 30691 ts_tablet_manager.cc:1888] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.527657 30691 log.cc:1196] T 8c157f7f72dc4211b256eee0015eed23 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/8c157f7f72dc4211b256eee0015eed23
I20240501 19:56:54.529561 26924 catalog_manager.cc:5484] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 reported cstate change: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.530257 26910 catalog_manager.cc:4830] TS 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155): tablet 8c157f7f72dc4211b256eee0015eed23 (table rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]) successfully deleted
I20240501 19:56:54.532397 27066 consensus_queue.cc:237] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:54.533591 27197 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.534016 30603 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.534085 29008 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.534387 30603 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.535387 30725 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.535712 27197 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.536955 26909 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6c498eacc5e947bfb01dfb90c7039df0 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.537496 26923 catalog_manager.cc:5484] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.537812 29008 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.540088 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c498eacc5e947bfb01dfb90c7039df0 with delete_type TABLET_DATA_TOMBSTONED (TS e5057264da53436ea17bad01c66a7fbf not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41962
I20240501 19:56:54.540316 30686 tablet_replica.cc:330] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf: stopping tablet replica
I20240501 19:56:54.540426 30686 raft_consensus.cc:2229] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.540565 30686 raft_consensus.cc:2258] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.541116 30686 ts_tablet_manager.cc:1875] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.542953 30686 ts_tablet_manager.cc:1888] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.543010 30686 log.cc:1196] T 6c498eacc5e947bfb01dfb90c7039df0 P e5057264da53436ea17bad01c66a7fbf: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/6c498eacc5e947bfb01dfb90c7039df0
I20240501 19:56:54.543416 26911 catalog_manager.cc:4830] TS e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115): tablet 6c498eacc5e947bfb01dfb90c7039df0 (table rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]) successfully deleted
I20240501 19:56:54.581539 27327 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6211064bb3ce41f8b19729d8d0de7fe8"
dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:35698
I20240501 19:56:54.581666 27327 raft_consensus.cc:595] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:56:54.611953 27327 consensus_queue.cc:237] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:54.613044 27197 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.613390 27066 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.613467 30735 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.613757 30121 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.615131 29433 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:54.615888 27197 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
W20240501 19:56:54.616798 27262 consensus_peers.cc:485] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529): Couldn't send request to peer 7ccb4fe66ab942d399551b10f5ed32ea. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e05a10bfc6540c19faa154896b8976a. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:54.617895 26915 catalog_manager.cc:5484] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.618208 27066 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:54.623411 30689 raft_consensus.cc:1050] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf: attempting to promote NON_VOTER 724bae28280242c6982c4f8e315b0d0b to VOTER
I20240501 19:56:54.623869 30689 consensus_queue.cc:237] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:56:54.625157 27066 raft_consensus.cc:1261] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.625155 27197 raft_consensus.cc:1261] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.625216 27327 consensus_queue.cc:237] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:54.625416 28746 raft_consensus.cc:1261] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.625603 30099 consensus_queue.cc:1035] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.625946 30099 consensus_queue.cc:1035] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.626696 27196 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.627198 30688 consensus_queue.cc:1035] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.627301 27457 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 19:56:54.627692 27263 consensus_peers.cc:485] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer 724bae28280242c6982c4f8e315b0d0b (127.20.210.5:42343): Couldn't send request to peer 724bae28280242c6982c4f8e315b0d0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 75cc96baf4654318a09dcc34eb32819c. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:54.627735 30737 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.627899 29433 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.628543 30685 raft_consensus.cc:2932] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.629462 30121 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:54.629972 27066 raft_consensus.cc:2932] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.630008 27196 raft_consensus.cc:2932] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.630522 27197 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:54.631448 27459 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:54.632393 26915 catalog_manager.cc:5484] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.634336 26922 catalog_manager.cc:5484] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index -1 to 4, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.634490 28746 raft_consensus.cc:2932] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.639568 27459 consensus_queue.cc:237] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:56:54.640524 28746 raft_consensus.cc:1261] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.640712 27197 raft_consensus.cc:1261] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.640857 30688 consensus_queue.cc:1035] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.641156 30099 consensus_queue.cc:1035] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.642431 30689 raft_consensus.cc:2932] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.642753 28746 raft_consensus.cc:2932] T e379c346ae524bd28f6e550051c0bf7e P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.643465 26911 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e379c346ae524bd28f6e550051c0bf7e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.644587 26922 catalog_manager.cc:5484] T e379c346ae524bd28f6e550051c0bf7e P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.644711 27197 raft_consensus.cc:2932] T e379c346ae524bd28f6e550051c0bf7e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.645604 27196 consensus_queue.cc:237] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:54.647095 27066 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.647553 30654 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 19:56:54.648005 27132 consensus_peers.cc:485] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 -> Peer ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695): Couldn't send request to peer ecdc0d100f5f4b95b9dda10c2f97cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 55d39ed3702046b08b1ac3c76c272c01. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:54.648201 27327 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.648550 30654 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.648708 27045 tablet_service.cc:1514] Processing DeleteTablet for tablet e379c346ae524bd28f6e550051c0bf7e with delete_type TABLET_DATA_TOMBSTONED (TS 16cfc5a5082742c5b09d35e5c4f7f10e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49644
I20240501 19:56:54.649190 30886 tablet_replica.cc:330] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e: stopping tablet replica
I20240501 19:56:54.649551 30886 raft_consensus.cc:2229] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.649570 30654 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:54.649760 30886 raft_consensus.cc:2258] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.651010 30886 ts_tablet_manager.cc:1875] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.651165 27327 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:54.651232 27066 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:54.653960 26923 catalog_manager.cc:5484] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:54.661329 27196 consensus_queue.cc:237] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:54.662809 27327 raft_consensus.cc:1261] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 19:56:54.663029 27132 consensus_peers.cc:485] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 -> Peer ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695): Couldn't send request to peer ecdc0d100f5f4b95b9dda10c2f97cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b3e84419a13348e4a0e7bacfe94cf33e. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:54.663182 27066 raft_consensus.cc:1261] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.663302 30654 consensus_queue.cc:1035] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.663575 30666 consensus_queue.cc:1035] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.665009 30654 raft_consensus.cc:2932] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:54.665416 27327 raft_consensus.cc:2932] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:54.668262 26922 catalog_manager.cc:5484] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.668434 30886 ts_tablet_manager.cc:1888] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.668509 30886 log.cc:1196] T e379c346ae524bd28f6e550051c0bf7e P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/e379c346ae524bd28f6e550051c0bf7e
I20240501 19:56:54.669165 26909 catalog_manager.cc:4830] TS 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375): tablet e379c346ae524bd28f6e550051c0bf7e (table rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]) successfully deleted
I20240501 19:56:54.669772 27066 raft_consensus.cc:2932] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:54.674300 27327 consensus_queue.cc:237] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:54.675431 27459 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.675590 27066 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 19:56:54.675693 27263 consensus_peers.cc:485] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695): Couldn't send request to peer ecdc0d100f5f4b95b9dda10c2f97cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 692732b291b441029826cf20c9a0ecb1. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:54.675730 29433 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.675935 30735 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.677587 30737 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:54.677949 27459 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:54.677985 27066 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:54.680385 26922 catalog_manager.cc:5484] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index -1 to 4, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:54.682112 30737 raft_consensus.cc:1050] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 to VOTER
I20240501 19:56:54.682548 30737 consensus_queue.cc:237] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:54.683626 27066 raft_consensus.cc:1261] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.683776 27196 raft_consensus.cc:1261] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.684258 30121 consensus_queue.cc:1035] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.684546 30121 consensus_queue.cc:1035] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.685667 29140 raft_consensus.cc:1261] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.686147 30735 consensus_queue.cc:1035] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.688616 30121 raft_consensus.cc:2932] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.689008 27196 raft_consensus.cc:2932] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.690328 27066 raft_consensus.cc:2932] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.690512 29140 raft_consensus.cc:2932] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.692186 26923 catalog_manager.cc:5484] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.696236 27327 consensus_queue.cc:237] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:54.698026 27197 raft_consensus.cc:1261] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.698110 29140 raft_consensus.cc:1261] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.698395 30735 consensus_queue.cc:1035] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.698572 29433 consensus_queue.cc:1035] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.700232 30737 raft_consensus.cc:2932] T 3c80c1dcbcd3439ba75066259557e716 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.700690 27197 raft_consensus.cc:2932] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.700755 29140 raft_consensus.cc:2932] T 3c80c1dcbcd3439ba75066259557e716 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.701604 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3c80c1dcbcd3439ba75066259557e716 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.702929 26915 catalog_manager.cc:5484] T 3c80c1dcbcd3439ba75066259557e716 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.705034 27045 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c80c1dcbcd3439ba75066259557e716 with delete_type TABLET_DATA_TOMBSTONED (TS 16cfc5a5082742c5b09d35e5c4f7f10e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49644
I20240501 19:56:54.705221 30886 tablet_replica.cc:330] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e: stopping tablet replica
I20240501 19:56:54.705366 30886 raft_consensus.cc:2229] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.705504 30886 raft_consensus.cc:2258] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.706159 30886 ts_tablet_manager.cc:1875] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.707903 30886 ts_tablet_manager.cc:1888] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.707968 30886 log.cc:1196] T 3c80c1dcbcd3439ba75066259557e716 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/3c80c1dcbcd3439ba75066259557e716
I20240501 19:56:54.708420 26909 catalog_manager.cc:4830] TS 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375): tablet 3c80c1dcbcd3439ba75066259557e716 (table rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]) successfully deleted
I20240501 19:56:54.730203 30121 raft_consensus.cc:1050] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 to VOTER
I20240501 19:56:54.730563 30121 consensus_queue.cc:237] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:54.731774 29008 raft_consensus.cc:1261] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.731773 27459 raft_consensus.cc:1261] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.731806 27197 raft_consensus.cc:1261] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.732184 30121 consensus_queue.cc:1035] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.732550 30121 consensus_queue.cc:1035] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.733577 29433 consensus_queue.cc:1035] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.734843 30735 raft_consensus.cc:2932] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.735251 27459 raft_consensus.cc:2932] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.737375 27197 raft_consensus.cc:2932] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.737421 26920 catalog_manager.cc:5484] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.739483 29008 raft_consensus.cc:2932] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.741514 27327 consensus_queue.cc:237] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:54.742790 27459 raft_consensus.cc:1261] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.743261 29008 raft_consensus.cc:1261] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.743232 30735 consensus_queue.cc:1035] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.743633 30735 consensus_queue.cc:1035] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.744827 30737 raft_consensus.cc:2932] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.745607 27459 raft_consensus.cc:2932] T e3b87a633efe4400931f1e0a566c8f14 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.745744 29008 raft_consensus.cc:2932] T e3b87a633efe4400931f1e0a566c8f14 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:54.746362 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e3b87a633efe4400931f1e0a566c8f14 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.747737 26922 catalog_manager.cc:5484] T e3b87a633efe4400931f1e0a566c8f14 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.750213 27175 tablet_service.cc:1514] Processing DeleteTablet for tablet e3b87a633efe4400931f1e0a566c8f14 with delete_type TABLET_DATA_TOMBSTONED (TS ef2d33d636574e8fbb9d3b90a15035d6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43438
I20240501 19:56:54.750422 30704 tablet_replica.cc:330] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6: stopping tablet replica
I20240501 19:56:54.750540 30704 raft_consensus.cc:2229] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.750667 30704 raft_consensus.cc:2258] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.751183 30704 ts_tablet_manager.cc:1875] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.752569 30704 ts_tablet_manager.cc:1888] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.752628 30704 log.cc:1196] T e3b87a633efe4400931f1e0a566c8f14 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/e3b87a633efe4400931f1e0a566c8f14
I20240501 19:56:54.753023 26908 catalog_manager.cc:4830] TS ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113): tablet e3b87a633efe4400931f1e0a566c8f14 (table rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]) successfully deleted
I20240501 19:56:54.769176 27327 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6211064bb3ce41f8b19729d8d0de7fe8"
dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:35784
I20240501 19:56:54.769261 27327 raft_consensus.cc:595] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:56:54.774190 30173 raft_consensus.cc:1050] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6: attempting to promote NON_VOTER 724bae28280242c6982c4f8e315b0d0b to VOTER
I20240501 19:56:54.774580 30173 consensus_queue.cc:237] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:56:54.775877 27327 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.776042 27459 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.776225 30657 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.776798 30657 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.777269 28746 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.777732 30657 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.779235 30655 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.779721 28746 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.781018 27459 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.781253 26920 catalog_manager.cc:5484] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.782058 27327 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.783807 27197 consensus_queue.cc:237] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:56:54.785497 27459 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.785944 30654 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.787281 28746 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.787788 30655 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.787750 30173 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.788161 27459 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.789024 26908 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0f417f4001bc48158187ed262f7737ab with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.789722 28746 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:54.790346 26915 catalog_manager.cc:5484] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.792222 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f417f4001bc48158187ed262f7737ab with delete_type TABLET_DATA_TOMBSTONED (TS 3c3f70fdad7b438a841f74fc638d17f5 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34348
I20240501 19:56:54.792383 30691 tablet_replica.cc:330] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5: stopping tablet replica
I20240501 19:56:54.792511 30691 raft_consensus.cc:2229] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.792625 30691 raft_consensus.cc:2258] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.793118 30691 ts_tablet_manager.cc:1875] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.793315 27327 consensus_queue.cc:237] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
W20240501 19:56:54.794623 27263 consensus_peers.cc:485] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8:44887): Couldn't send request to peer 474eb4491dcb47f4a2a2f6e149bc39c1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5a9674523a2148b1a6c1a5d22a57aac5. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:54.794965 30691 ts_tablet_manager.cc:1888] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.795039 30691 log.cc:1196] T 0f417f4001bc48158187ed262f7737ab P 3c3f70fdad7b438a841f74fc638d17f5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/0f417f4001bc48158187ed262f7737ab
I20240501 19:56:54.795192 27197 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.795429 26910 catalog_manager.cc:4830] TS 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155): tablet 0f417f4001bc48158187ed262f7737ab (table rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]) successfully deleted
I20240501 19:56:54.795497 30657 raft_consensus.cc:1050] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6: attempting to promote NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea to VOTER
I20240501 19:56:54.795511 27459 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.795792 30657 consensus_queue.cc:237] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:54.796720 29433 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.797000 30121 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.797035 28877 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.797108 27066 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.797430 27459 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.797542 30655 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.797804 30654 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.798067 30654 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.798851 30735 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.799181 27197 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.800927 28877 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.801077 30657 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.801725 26922 catalog_manager.cc:5484] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.802728 27457 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.803512 26915 catalog_manager.cc:5484] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea reported cstate change: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.804745 27066 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.806772 27457 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.812250 30725 raft_consensus.cc:1050] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e: attempting to promote NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 to VOTER
I20240501 19:56:54.812402 27197 consensus_queue.cc:237] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:54.812562 30725 consensus_queue.cc:237] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:54.813402 28877 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.813732 30655 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.813853 27457 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.813865 29140 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.813865 27327 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.814070 27459 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.814241 29484 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.814524 29484 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.814715 30725 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.814824 30655 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.815537 30655 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.816522 27457 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.817135 26908 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b2c944ca787b475294f9c41f6052a7ea with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.817085 28877 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:54.818598 27326 consensus_queue.cc:237] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:54.819015 26923 catalog_manager.cc:5484] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.820008 27066 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.820029 27458 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.820345 30735 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.821224 27045 tablet_service.cc:1514] Processing DeleteTablet for tablet b2c944ca787b475294f9c41f6052a7ea with delete_type TABLET_DATA_TOMBSTONED (TS 16cfc5a5082742c5b09d35e5c4f7f10e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49644
I20240501 19:56:54.821329 30735 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.821383 30886 tablet_replica.cc:330] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e: stopping tablet replica
I20240501 19:56:54.821496 30886 raft_consensus.cc:2229] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.821616 30886 raft_consensus.cc:2258] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.821630 29484 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
W20240501 19:56:54.821933 27262 consensus_peers.cc:485] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529): Couldn't send request to peer 7ccb4fe66ab942d399551b10f5ed32ea. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0723850372f8416d9b8ac19f119939bc. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:54.821987 27327 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.822333 30886 ts_tablet_manager.cc:1875] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.824285 30886 ts_tablet_manager.cc:1888] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.824357 30886 log.cc:1196] T b2c944ca787b475294f9c41f6052a7ea P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/b2c944ca787b475294f9c41f6052a7ea
I20240501 19:56:54.824432 26923 catalog_manager.cc:5484] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.824852 26909 catalog_manager.cc:4830] TS 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375): tablet b2c944ca787b475294f9c41f6052a7ea (table rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]) successfully deleted
I20240501 19:56:54.825549 27459 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.826332 30121 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:54.827332 27064 consensus_queue.cc:237] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:54.827630 29140 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.828899 27327 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.829389 27459 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:54.829726 29140 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.829655 26922 catalog_manager.cc:5484] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.830719 27066 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:54.831777 30603 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.832079 30603 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.834925 30725 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.835613 27327 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.836566 29140 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.837046 26909 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6aff1d7fd2d2476095ef0a0529110b49 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.837841 26922 catalog_manager.cc:5484] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.839716 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aff1d7fd2d2476095ef0a0529110b49 with delete_type TABLET_DATA_TOMBSTONED (TS e5057264da53436ea17bad01c66a7fbf not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41962
I20240501 19:56:54.839918 30686 tablet_replica.cc:330] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf: stopping tablet replica
I20240501 19:56:54.840036 30686 raft_consensus.cc:2229] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.840149 30686 raft_consensus.cc:2258] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.840173 27327 consensus_queue.cc:237] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:54.840694 30686 ts_tablet_manager.cc:1875] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.841506 27459 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.841584 27066 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 19:56:54.841779 27262 consensus_peers.cc:485] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529): Couldn't send request to peer 7ccb4fe66ab942d399551b10f5ed32ea. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0aaa59df03844aadafb8d57797685dc2. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:54.841792 29433 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.842106 29433 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.842876 30686 ts_tablet_manager.cc:1888] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.842962 30686 log.cc:1196] T 6aff1d7fd2d2476095ef0a0529110b49 P e5057264da53436ea17bad01c66a7fbf: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/6aff1d7fd2d2476095ef0a0529110b49
I20240501 19:56:54.843519 26911 catalog_manager.cc:4830] TS e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115): tablet 6aff1d7fd2d2476095ef0a0529110b49 (table rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]) successfully deleted
I20240501 19:56:54.843593 30735 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:54.844097 27066 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:54.845854 27459 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:54.846081 26923 catalog_manager.cc:5484] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index -1 to 4, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.862967 30737 raft_consensus.cc:1050] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 to VOTER
I20240501 19:56:54.863209 30737 consensus_queue.cc:237] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:54.864482 27066 raft_consensus.cc:1261] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.864548 29140 raft_consensus.cc:1261] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.864560 27459 raft_consensus.cc:1261] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.864830 30737 consensus_queue.cc:1035] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.865104 29433 consensus_queue.cc:1035] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.865617 30737 consensus_queue.cc:1035] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.866828 30737 raft_consensus.cc:2932] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.867170 27459 raft_consensus.cc:2932] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.868947 27066 raft_consensus.cc:2932] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.869211 26915 catalog_manager.cc:5484] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.870572 30685 raft_consensus.cc:1050] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf: attempting to promote NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 to VOTER
I20240501 19:56:54.870851 30685 consensus_queue.cc:237] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:54.872184 29140 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.872341 27066 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.872424 29139 raft_consensus.cc:2932] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.874300 30099 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.875572 30099 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.877522 27327 consensus_queue.cc:237] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:54.878103 27326 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:54.878423 30099 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:54.878499 29140 raft_consensus.cc:1261] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.878767 29433 consensus_queue.cc:1035] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.878816 27459 raft_consensus.cc:1261] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.879101 29433 consensus_queue.cc:1035] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.879880 30099 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.880273 29140 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.880764 29433 raft_consensus.cc:979] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5: : Instructing follower ecdc0d100f5f4b95b9dda10c2f97cf20 to start an election
I20240501 19:56:54.880851 29433 raft_consensus.cc:1067] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Signalling peer ecdc0d100f5f4b95b9dda10c2f97cf20 to start an election
I20240501 19:56:54.881094 29433 raft_consensus.cc:2932] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.882128 27066 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.882158 26915 catalog_manager.cc:5484] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.882670 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5ecba34e358c41d9b812113c955b6b29 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.883241 29008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6211064bb3ce41f8b19729d8d0de7fe8"
dest_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20"
from {username='slave'} at 127.20.210.3:44511
I20240501 19:56:54.883399 29008 raft_consensus.cc:482] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 19:56:54.883507 29008 raft_consensus.cc:3037] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:56:54.884348 26923 catalog_manager.cc:5484] T 5ecba34e358c41d9b812113c955b6b29 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.884513 29008 raft_consensus.cc:504] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:54.885118 27459 raft_consensus.cc:2932] T 5ecba34e358c41d9b812113c955b6b29 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.885751 29008 leader_election.cc:288] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [CANDIDATE]: Term 2 election: Requested vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:54.887059 27045 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ecba34e358c41d9b812113c955b6b29 with delete_type TABLET_DATA_TOMBSTONED (TS 16cfc5a5082742c5b09d35e5c4f7f10e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49644
I20240501 19:56:54.887243 30886 tablet_replica.cc:330] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e: stopping tablet replica
I20240501 19:56:54.887390 30886 raft_consensus.cc:2229] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.887501 30886 raft_consensus.cc:2258] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.887670 27197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6211064bb3ce41f8b19729d8d0de7fe8" candidate_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:54.887741 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6211064bb3ce41f8b19729d8d0de7fe8" candidate_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:56:54.887773 27197 raft_consensus.cc:3037] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:56:54.887818 27066 raft_consensus.cc:3037] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:56:54.888522 27066 raft_consensus.cc:2454] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecdc0d100f5f4b95b9dda10c2f97cf20 in term 2.
I20240501 19:56:54.888669 27197 raft_consensus.cc:2454] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecdc0d100f5f4b95b9dda10c2f97cf20 in term 2.
I20240501 19:56:54.889228 30886 ts_tablet_manager.cc:1875] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.889580 28941 leader_election.cc:302] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, ecdc0d100f5f4b95b9dda10c2f97cf20, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:56:54.889794 30462 raft_consensus.cc:2781] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 19:56:54.889951 30462 raft_consensus.cc:686] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 LEADER]: Becoming Leader. State: Replica: ecdc0d100f5f4b95b9dda10c2f97cf20, State: Running, Role: LEADER
I20240501 19:56:54.890327 30462 consensus_queue.cc:237] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:54.890967 27459 consensus_queue.cc:237] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:54.891175 30886 ts_tablet_manager.cc:1888] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.891230 30886 log.cc:1196] T 5ecba34e358c41d9b812113c955b6b29 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/5ecba34e358c41d9b812113c955b6b29
I20240501 19:56:54.891883 29140 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.892154 30099 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.892757 26909 catalog_manager.cc:4830] TS 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375): tablet 5ecba34e358c41d9b812113c955b6b29 (table rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]) successfully deleted
I20240501 19:56:54.892830 26923 catalog_manager.cc:5484] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 reported cstate change: term changed from 1 to 2, leader changed from 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) to ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7). New cstate: current_term: 2 leader_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.893448 27327 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.893702 27326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6211064bb3ce41f8b19729d8d0de7fe8" candidate_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:54.893807 27326 raft_consensus.cc:3032] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 19:56:54.893847 27326 raft_consensus.cc:729] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:54.893980 27326 consensus_queue.cc:260] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:54.894151 27326 raft_consensus.cc:3037] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:56:54.894542 29140 raft_consensus.cc:2932] T 5ecba34e358c41d9b812113c955b6b29 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.894655 27328 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:54.895046 27326 raft_consensus.cc:2454] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecdc0d100f5f4b95b9dda10c2f97cf20 in term 2.
I20240501 19:56:54.895435 30689 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:54.896932 30099 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.897924 27326 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.898774 26911 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9102f7c8350843c2bd7009ea190860d0 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:54.899344 29140 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:54.899773 26923 catalog_manager.cc:5484] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:54.902181 27045 tablet_service.cc:1514] Processing DeleteTablet for tablet 9102f7c8350843c2bd7009ea190860d0 with delete_type TABLET_DATA_TOMBSTONED (TS 16cfc5a5082742c5b09d35e5c4f7f10e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49644
I20240501 19:56:54.902380 30886 tablet_replica.cc:330] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e: stopping tablet replica
I20240501 19:56:54.902499 30886 raft_consensus.cc:2229] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:54.902627 30886 raft_consensus.cc:2258] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:54.903826 30886 ts_tablet_manager.cc:1875] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:54.905269 30886 ts_tablet_manager.cc:1888] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:54.905332 30886 log.cc:1196] T 9102f7c8350843c2bd7009ea190860d0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/9102f7c8350843c2bd7009ea190860d0
I20240501 19:56:54.905733 26909 catalog_manager.cc:4830] TS 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375): tablet 9102f7c8350843c2bd7009ea190860d0 (table rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]) successfully deleted
I20240501 19:56:54.960572 27459 consensus_queue.cc:237] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:54.961695 27066 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.961863 27197 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 19:56:54.961926 27394 consensus_peers.cc:485] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf -> Peer 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8:44887): Couldn't send request to peer 474eb4491dcb47f4a2a2f6e149bc39c1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ca822a1462544c3be45371d6e088447. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:54.962157 30099 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.962345 30688 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.963699 30099 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.964570 27066 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.965130 27197 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.966398 26923 catalog_manager.cc:5484] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:54.976141 27066 consensus_queue.cc:237] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:54.977301 27197 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.977510 27326 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.977695 30727 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 19:56:54.978101 27001 consensus_peers.cc:485] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e -> Peer 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8:44887): Couldn't send request to peer 474eb4491dcb47f4a2a2f6e149bc39c1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c9f961f0554c45a1a3f31d5a8b31a46d. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:54.978263 30727 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.979346 30603 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.979672 27197 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.981701 26922 catalog_manager.cc:5484] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.989091 27459 consensus_queue.cc:237] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:54.990301 27197 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:56:54.990350 27328 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 19:56:54.990502 27394 consensus_peers.cc:485] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf -> Peer 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8:44887): Couldn't send request to peer 474eb4491dcb47f4a2a2f6e149bc39c1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6ee4db743833421e8120edbec4ba0836. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:54.990676 30688 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.990861 30685 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:56:54.992391 30689 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.992712 27328 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.995177 26922 catalog_manager.cc:5484] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.996371 30173 raft_consensus.cc:1050] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6: attempting to promote NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 to VOTER
I20240501 19:56:54.996694 30173 consensus_queue.cc:237] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:54.997849 27326 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:54.998633 27197 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:55.000101 29140 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:55.001380 27328 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:55.001435 27459 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:55.001760 30654 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:55.001935 30655 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:55.003194 30655 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:55.006861 31151 ts_tablet_manager.cc:902] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:55.007411 31151 tablet_copy_client.cc:308] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:55.008646 30655 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:55.009907 29140 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:55.010604 27459 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:55.011921 26920 catalog_manager.cc:5484] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:55.012284 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 3e05a10bfc6540c19faa154896b8976a from peer 7ccb4fe66ab942d399551b10f5ed32ea ({username='slave'} at 127.20.210.6:59627)
I20240501 19:56:55.012363 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 3e05a10bfc6540c19faa154896b8976a from peer 7ccb4fe66ab942d399551b10f5ed32ea at {username='slave'} at 127.20.210.6:59627: session id = 7ccb4fe66ab942d399551b10f5ed32ea-3e05a10bfc6540c19faa154896b8976a
I20240501 19:56:55.013186 27348 tablet_copy_source_session.cc:205] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:55.013378 27328 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:55.013854 31151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e05a10bfc6540c19faa154896b8976a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:55.016054 27197 consensus_queue.cc:237] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:55.017230 27459 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:55.017249 29140 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:55.017550 30173 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:55.017836 30173 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:55.019177 30173 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:55.019618 27459 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:55.020207 29140 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:55.020476 26908 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 896072a285b649dbb31f9f02666ace25 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:55.020959 31151 tablet_copy_client.cc:783] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:55.021200 31151 tablet_copy_client.cc:647] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:55.021983 26922 catalog_manager.cc:5484] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:55.022253 31157 tablet_copy_client.cc:1002] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:55.024201 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet 896072a285b649dbb31f9f02666ace25 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3f70fdad7b438a841f74fc638d17f5 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34348
I20240501 19:56:55.024392 30691 tablet_replica.cc:330] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5: stopping tablet replica
I20240501 19:56:55.024509 30691 raft_consensus.cc:2229] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:55.024633 30691 raft_consensus.cc:2258] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:55.025367 30691 ts_tablet_manager.cc:1875] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:55.027384 30691 ts_tablet_manager.cc:1888] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:55.027458 30691 log.cc:1196] T 896072a285b649dbb31f9f02666ace25 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/896072a285b649dbb31f9f02666ace25
I20240501 19:56:55.027895 26910 catalog_manager.cc:4830] TS 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155): tablet 896072a285b649dbb31f9f02666ace25 (table rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]) successfully deleted
I20240501 19:56:55.111768 31262 ts_tablet_manager.cc:902] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:55.112321 31262 tablet_copy_client.cc:308] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:55.112820 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 692732b291b441029826cf20c9a0ecb1 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 ({username='slave'} at 127.20.210.7:35965)
I20240501 19:56:55.112907 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 692732b291b441029826cf20c9a0ecb1 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 at {username='slave'} at 127.20.210.7:35965: session id = ecdc0d100f5f4b95b9dda10c2f97cf20-692732b291b441029826cf20c9a0ecb1
I20240501 19:56:55.113688 27348 tablet_copy_source_session.cc:205] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:55.114545 31262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692732b291b441029826cf20c9a0ecb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:55.116830 31262 tablet_copy_client.cc:783] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:55.116976 31262 tablet_copy_client.cc:647] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:55.118209 31266 tablet_copy_client.cc:1002] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:55.172757 31282 ts_tablet_manager.cc:902] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:55.173252 31282 tablet_copy_client.cc:308] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:55.173707 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 75cc96baf4654318a09dcc34eb32819c from peer 724bae28280242c6982c4f8e315b0d0b ({username='slave'} at 127.20.210.5:40583)
I20240501 19:56:55.173799 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 75cc96baf4654318a09dcc34eb32819c from peer 724bae28280242c6982c4f8e315b0d0b at {username='slave'} at 127.20.210.5:40583: session id = 724bae28280242c6982c4f8e315b0d0b-75cc96baf4654318a09dcc34eb32819c
I20240501 19:56:55.174525 27348 tablet_copy_source_session.cc:205] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:55.175256 31282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75cc96baf4654318a09dcc34eb32819c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:55.176891 31282 tablet_copy_client.cc:783] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:55.177057 31282 tablet_copy_client.cc:647] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:55.177979 31286 tablet_copy_client.cc:1002] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:55.190300 31287 ts_tablet_manager.cc:902] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20: Initiating tablet copy from peer ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:55.190737 31287 tablet_copy_client.cc:308] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:43113
I20240501 19:56:55.191203 27217 tablet_copy_service.cc:140] P ef2d33d636574e8fbb9d3b90a15035d6: Received BeginTabletCopySession request for tablet 55d39ed3702046b08b1ac3c76c272c01 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 ({username='slave'} at 127.20.210.7:46547)
I20240501 19:56:55.191290 27217 tablet_copy_service.cc:161] P ef2d33d636574e8fbb9d3b90a15035d6: Beginning new tablet copy session on tablet 55d39ed3702046b08b1ac3c76c272c01 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 at {username='slave'} at 127.20.210.7:46547: session id = ecdc0d100f5f4b95b9dda10c2f97cf20-55d39ed3702046b08b1ac3c76c272c01
I20240501 19:56:55.192013 27217 tablet_copy_source_session.cc:205] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:55.192807 31287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d39ed3702046b08b1ac3c76c272c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:55.194283 31287 tablet_copy_client.cc:783] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:55.194422 31287 tablet_copy_client.cc:647] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:55.195360 31290 tablet_copy_client.cc:1002] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:55.205888 31292 ts_tablet_manager.cc:902] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:55.206403 31292 tablet_copy_client.cc:308] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:55.206831 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 0723850372f8416d9b8ac19f119939bc from peer 7ccb4fe66ab942d399551b10f5ed32ea ({username='slave'} at 127.20.210.6:59627)
I20240501 19:56:55.206909 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 0723850372f8416d9b8ac19f119939bc from peer 7ccb4fe66ab942d399551b10f5ed32ea at {username='slave'} at 127.20.210.6:59627: session id = 7ccb4fe66ab942d399551b10f5ed32ea-0723850372f8416d9b8ac19f119939bc
I20240501 19:56:55.207486 27348 tablet_copy_source_session.cc:205] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:55.208153 31292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0723850372f8416d9b8ac19f119939bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:55.209673 31292 tablet_copy_client.cc:783] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:55.209820 31292 tablet_copy_client.cc:647] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:55.210716 31296 tablet_copy_client.cc:1002] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:55.228332 31297 ts_tablet_manager.cc:902] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20: Initiating tablet copy from peer ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:55.228704 31297 tablet_copy_client.cc:308] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:43113
I20240501 19:56:55.229066 27217 tablet_copy_service.cc:140] P ef2d33d636574e8fbb9d3b90a15035d6: Received BeginTabletCopySession request for tablet b3e84419a13348e4a0e7bacfe94cf33e from peer ecdc0d100f5f4b95b9dda10c2f97cf20 ({username='slave'} at 127.20.210.7:46547)
I20240501 19:56:55.229142 27217 tablet_copy_service.cc:161] P ef2d33d636574e8fbb9d3b90a15035d6: Beginning new tablet copy session on tablet b3e84419a13348e4a0e7bacfe94cf33e from peer ecdc0d100f5f4b95b9dda10c2f97cf20 at {username='slave'} at 127.20.210.7:46547: session id = ecdc0d100f5f4b95b9dda10c2f97cf20-b3e84419a13348e4a0e7bacfe94cf33e
I20240501 19:56:55.229698 27217 tablet_copy_source_session.cc:205] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:55.230355 31297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3e84419a13348e4a0e7bacfe94cf33e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:55.231858 31297 tablet_copy_client.cc:783] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:55.232000 31297 tablet_copy_client.cc:647] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:55.232816 31301 tablet_copy_client.cc:1002] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:55.336555 31311 ts_tablet_manager.cc:902] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:55.337086 31311 tablet_copy_client.cc:308] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:55.337545 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 5a9674523a2148b1a6c1a5d22a57aac5 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 ({username='slave'} at 127.20.210.8:56643)
I20240501 19:56:55.337635 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 5a9674523a2148b1a6c1a5d22a57aac5 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 at {username='slave'} at 127.20.210.8:56643: session id = 474eb4491dcb47f4a2a2f6e149bc39c1-5a9674523a2148b1a6c1a5d22a57aac5
I20240501 19:56:55.338302 27348 tablet_copy_source_session.cc:205] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:55.339073 31311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a9674523a2148b1a6c1a5d22a57aac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:55.341353 31311 tablet_copy_client.cc:783] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:55.341533 31311 tablet_copy_client.cc:647] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:55.343470 31315 tablet_copy_client.cc:1002] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:55.364357 27066 raft_consensus.cc:1261] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 19:56:55.364822 31066 consensus_queue.cc:1035] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:55.369045 31332 ts_tablet_manager.cc:902] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:55.369581 31332 tablet_copy_client.cc:308] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:55.369961 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 0aaa59df03844aadafb8d57797685dc2 from peer 7ccb4fe66ab942d399551b10f5ed32ea ({username='slave'} at 127.20.210.6:59627)
I20240501 19:56:55.370061 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 0aaa59df03844aadafb8d57797685dc2 from peer 7ccb4fe66ab942d399551b10f5ed32ea at {username='slave'} at 127.20.210.6:59627: session id = 7ccb4fe66ab942d399551b10f5ed32ea-0aaa59df03844aadafb8d57797685dc2
I20240501 19:56:55.370820 27348 tablet_copy_source_session.cc:205] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:55.371583 31332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aaa59df03844aadafb8d57797685dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:55.373487 31332 tablet_copy_client.cc:783] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:55.373625 31332 tablet_copy_client.cc:647] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:55.374009 27197 raft_consensus.cc:1261] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 19:56:55.374351 31066 consensus_queue.cc:1035] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:55.374802 31335 tablet_copy_client.cc:1002] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:55.377038 27328 raft_consensus.cc:1261] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 19:56:55.379524 31347 consensus_queue.cc:1035] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:55.383589 29008 consensus_queue.cc:237] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:55.384774 27197 raft_consensus.cc:1261] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 19:56:55.384994 27066 raft_consensus.cc:1261] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 19:56:55.385308 30462 consensus_queue.cc:1035] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:55.385468 31347 consensus_queue.cc:1035] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:55.387286 31358 ts_tablet_manager.cc:902] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1: Initiating tablet copy from peer e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:55.387315 30462 raft_consensus.cc:2932] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:55.387609 27066 raft_consensus.cc:2932] T 6211064bb3ce41f8b19729d8d0de7fe8 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:55.388037 31358 tablet_copy_client.cc:308] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:37115
I20240501 19:56:55.388440 27479 tablet_copy_service.cc:140] P e5057264da53436ea17bad01c66a7fbf: Received BeginTabletCopySession request for tablet 6ee4db743833421e8120edbec4ba0836 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 ({username='slave'} at 127.20.210.8:39999)
I20240501 19:56:55.388536 27479 tablet_copy_service.cc:161] P e5057264da53436ea17bad01c66a7fbf: Beginning new tablet copy session on tablet 6ee4db743833421e8120edbec4ba0836 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 at {username='slave'} at 127.20.210.8:39999: session id = 474eb4491dcb47f4a2a2f6e149bc39c1-6ee4db743833421e8120edbec4ba0836
I20240501 19:56:55.388749 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6211064bb3ce41f8b19729d8d0de7fe8 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:55.389499 27479 tablet_copy_source_session.cc:205] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:55.389488 27197 raft_consensus.cc:2932] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:55.390370 31358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee4db743833421e8120edbec4ba0836. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:55.392449 31358 tablet_copy_client.cc:783] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:55.392567 31358 tablet_copy_client.cc:647] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:55.393179 26923 catalog_manager.cc:5484] T 6211064bb3ce41f8b19729d8d0de7fe8 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 5 to 7, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New cstate: current_term: 2 leader_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:55.393474 31360 tablet_copy_client.cc:1002] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:55.395359 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet 6211064bb3ce41f8b19729d8d0de7fe8 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3f70fdad7b438a841f74fc638d17f5 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:34348
I20240501 19:56:55.395560 30691 tablet_replica.cc:330] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5: stopping tablet replica
I20240501 19:56:55.395682 30691 raft_consensus.cc:2229] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:55.395776 30691 raft_consensus.cc:2258] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:55.396579 30691 ts_tablet_manager.cc:1875] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:55.398815 30691 ts_tablet_manager.cc:1888] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240501 19:56:55.398891 30691 log.cc:1196] T 6211064bb3ce41f8b19729d8d0de7fe8 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/6211064bb3ce41f8b19729d8d0de7fe8
I20240501 19:56:55.399298 26910 catalog_manager.cc:4830] TS 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155): tablet 6211064bb3ce41f8b19729d8d0de7fe8 (table rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]) successfully deleted
I20240501 19:56:55.560127 31439 ts_tablet_manager.cc:902] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1: Initiating tablet copy from peer 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:55.560570 31439 tablet_copy_client.cc:308] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:33375
I20240501 19:56:55.561029 27085 tablet_copy_service.cc:140] P 16cfc5a5082742c5b09d35e5c4f7f10e: Received BeginTabletCopySession request for tablet c9f961f0554c45a1a3f31d5a8b31a46d from peer 474eb4491dcb47f4a2a2f6e149bc39c1 ({username='slave'} at 127.20.210.8:42289)
I20240501 19:56:55.561122 27085 tablet_copy_service.cc:161] P 16cfc5a5082742c5b09d35e5c4f7f10e: Beginning new tablet copy session on tablet c9f961f0554c45a1a3f31d5a8b31a46d from peer 474eb4491dcb47f4a2a2f6e149bc39c1 at {username='slave'} at 127.20.210.8:42289: session id = 474eb4491dcb47f4a2a2f6e149bc39c1-c9f961f0554c45a1a3f31d5a8b31a46d
I20240501 19:56:55.561807 27085 tablet_copy_source_session.cc:205] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:55.562651 31439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9f961f0554c45a1a3f31d5a8b31a46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:55.563761 31444 ts_tablet_manager.cc:902] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1: Initiating tablet copy from peer e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:55.564208 31444 tablet_copy_client.cc:308] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:37115
I20240501 19:56:55.564586 31439 tablet_copy_client.cc:783] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:55.564648 27479 tablet_copy_service.cc:140] P e5057264da53436ea17bad01c66a7fbf: Received BeginTabletCopySession request for tablet 3ca822a1462544c3be45371d6e088447 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 ({username='slave'} at 127.20.210.8:39999)
I20240501 19:56:55.564731 31439 tablet_copy_client.cc:647] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:55.564752 27479 tablet_copy_service.cc:161] P e5057264da53436ea17bad01c66a7fbf: Beginning new tablet copy session on tablet 3ca822a1462544c3be45371d6e088447 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 at {username='slave'} at 127.20.210.8:39999: session id = 474eb4491dcb47f4a2a2f6e149bc39c1-3ca822a1462544c3be45371d6e088447
I20240501 19:56:55.565510 27479 tablet_copy_source_session.cc:205] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:55.565676 31443 tablet_copy_client.cc:1002] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:55.566183 31444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca822a1462544c3be45371d6e088447. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:55.567744 31444 tablet_copy_client.cc:783] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:55.567873 31444 tablet_copy_client.cc:647] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:55.568698 31448 tablet_copy_client.cc:1002] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:56.522572 31151 tablet_copy_client.cc:523] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:56.523981 31151 tablet_bootstrap.cc:492] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap starting.
I20240501 19:56:56.540514 31151 tablet_bootstrap.cc:492] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:56.540722 31151 tablet_bootstrap.cc:492] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap complete.
I20240501 19:56:56.540848 31151 ts_tablet_manager.cc:1367] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.004s
I20240501 19:56:56.541131 31151 raft_consensus.cc:348] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:56.541220 31151 raft_consensus.cc:729] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7ccb4fe66ab942d399551b10f5ed32ea, State: Initialized, Role: LEARNER
I20240501 19:56:56.541290 31151 consensus_queue.cc:260] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:56.541513 31151 ts_tablet_manager.cc:1398] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent starting tablet: real 0.001s user 0.003s sys 0.001s
I20240501 19:56:56.541954 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-3e05a10bfc6540c19faa154896b8976a received from {username='slave'} at 127.20.210.6:59627
I20240501 19:56:56.542034 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-3e05a10bfc6540c19faa154896b8976a on tablet 3e05a10bfc6540c19faa154896b8976a with peer 7ccb4fe66ab942d399551b10f5ed32ea
I20240501 19:56:56.618534 31262 tablet_copy_client.cc:523] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:56.619840 31262 tablet_bootstrap.cc:492] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap starting.
I20240501 19:56:56.624526 28877 raft_consensus.cc:1203] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:56.636796 31262 tablet_bootstrap.cc:492] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:56.637004 31262 tablet_bootstrap.cc:492] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap complete.
I20240501 19:56:56.637158 31262 ts_tablet_manager.cc:1367] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent bootstrapping tablet: real 0.017s user 0.016s sys 0.000s
I20240501 19:56:56.637483 31262 raft_consensus.cc:348] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:56.637579 31262 raft_consensus.cc:729] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ecdc0d100f5f4b95b9dda10c2f97cf20, State: Initialized, Role: LEARNER
I20240501 19:56:56.637655 31262 consensus_queue.cc:260] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:56.637893 31262 ts_tablet_manager.cc:1398] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:56.638401 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-692732b291b441029826cf20c9a0ecb1 received from {username='slave'} at 127.20.210.7:35965
I20240501 19:56:56.638495 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-692732b291b441029826cf20c9a0ecb1 on tablet 692732b291b441029826cf20c9a0ecb1 with peer ecdc0d100f5f4b95b9dda10c2f97cf20
I20240501 19:56:56.678344 31282 tablet_copy_client.cc:523] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:56.679896 31282 tablet_bootstrap.cc:492] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b: Bootstrap starting.
I20240501 19:56:56.695629 31287 tablet_copy_client.cc:523] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:56.697209 31287 tablet_bootstrap.cc:492] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap starting.
I20240501 19:56:56.697899 31282 tablet_bootstrap.cc:492] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:56.698165 31282 tablet_bootstrap.cc:492] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b: Bootstrap complete.
I20240501 19:56:56.698354 31282 ts_tablet_manager.cc:1367] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b: Time spent bootstrapping tablet: real 0.019s user 0.019s sys 0.000s
I20240501 19:56:56.698711 31282 raft_consensus.cc:348] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:56.698812 31282 raft_consensus.cc:729] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 724bae28280242c6982c4f8e315b0d0b, State: Initialized, Role: LEARNER
I20240501 19:56:56.698930 31282 consensus_queue.cc:260] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:56.699223 31282 ts_tablet_manager.cc:1398] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:56.699661 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session 724bae28280242c6982c4f8e315b0d0b-75cc96baf4654318a09dcc34eb32819c received from {username='slave'} at 127.20.210.5:40583
I20240501 19:56:56.699748 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session 724bae28280242c6982c4f8e315b0d0b-75cc96baf4654318a09dcc34eb32819c on tablet 75cc96baf4654318a09dcc34eb32819c with peer 724bae28280242c6982c4f8e315b0d0b
I20240501 19:56:56.711000 31292 tablet_copy_client.cc:523] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:56.712596 31292 tablet_bootstrap.cc:492] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap starting.
I20240501 19:56:56.720958 31287 tablet_bootstrap.cc:492] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:56.721220 31287 tablet_bootstrap.cc:492] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap complete.
I20240501 19:56:56.721383 31287 ts_tablet_manager.cc:1367] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.009s
I20240501 19:56:56.721735 31287 raft_consensus.cc:348] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:56.721837 31287 raft_consensus.cc:729] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ecdc0d100f5f4b95b9dda10c2f97cf20, State: Initialized, Role: LEARNER
I20240501 19:56:56.721931 31287 consensus_queue.cc:260] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:56.722194 31287 ts_tablet_manager.cc:1398] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:56.722914 27217 tablet_copy_service.cc:342] P ef2d33d636574e8fbb9d3b90a15035d6: Request end of tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-55d39ed3702046b08b1ac3c76c272c01 received from {username='slave'} at 127.20.210.7:46547
I20240501 19:56:56.723003 27217 tablet_copy_service.cc:433] P ef2d33d636574e8fbb9d3b90a15035d6: ending tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-55d39ed3702046b08b1ac3c76c272c01 on tablet 55d39ed3702046b08b1ac3c76c272c01 with peer ecdc0d100f5f4b95b9dda10c2f97cf20
I20240501 19:56:56.733031 31297 tablet_copy_client.cc:523] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:56.733274 31292 tablet_bootstrap.cc:492] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:56.733537 31292 tablet_bootstrap.cc:492] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap complete.
I20240501 19:56:56.733693 31292 ts_tablet_manager.cc:1367] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent bootstrapping tablet: real 0.021s user 0.020s sys 0.000s
I20240501 19:56:56.734025 31292 raft_consensus.cc:348] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:56.734118 31292 raft_consensus.cc:729] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7ccb4fe66ab942d399551b10f5ed32ea, State: Initialized, Role: LEARNER
I20240501 19:56:56.734174 31297 tablet_bootstrap.cc:492] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap starting.
I20240501 19:56:56.734206 31292 consensus_queue.cc:260] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:56.734500 31292 ts_tablet_manager.cc:1398] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:56.735386 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-0723850372f8416d9b8ac19f119939bc received from {username='slave'} at 127.20.210.6:59627
I20240501 19:56:56.735460 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-0723850372f8416d9b8ac19f119939bc on tablet 0723850372f8416d9b8ac19f119939bc with peer 7ccb4fe66ab942d399551b10f5ed32ea
I20240501 19:56:56.756623 31297 tablet_bootstrap.cc:492] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:56.756929 31297 tablet_bootstrap.cc:492] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap complete.
I20240501 19:56:56.757090 31297 ts_tablet_manager.cc:1367] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent bootstrapping tablet: real 0.023s user 0.021s sys 0.000s
I20240501 19:56:56.757485 31297 raft_consensus.cc:348] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:56.757577 31297 raft_consensus.cc:729] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ecdc0d100f5f4b95b9dda10c2f97cf20, State: Initialized, Role: LEARNER
I20240501 19:56:56.757652 31297 consensus_queue.cc:260] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:56.757917 31297 ts_tablet_manager.cc:1398] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 19:56:56.758358 27217 tablet_copy_service.cc:342] P ef2d33d636574e8fbb9d3b90a15035d6: Request end of tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-b3e84419a13348e4a0e7bacfe94cf33e received from {username='slave'} at 127.20.210.7:46547
I20240501 19:56:56.758431 27217 tablet_copy_service.cc:433] P ef2d33d636574e8fbb9d3b90a15035d6: ending tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-b3e84419a13348e4a0e7bacfe94cf33e on tablet b3e84419a13348e4a0e7bacfe94cf33e with peer ecdc0d100f5f4b95b9dda10c2f97cf20
I20240501 19:56:56.843712 31311 tablet_copy_client.cc:523] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:56.845028 31311 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap starting.
I20240501 19:56:56.860036 31311 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:56.860248 31311 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap complete.
I20240501 19:56:56.860416 31311 ts_tablet_manager.cc:1367] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent bootstrapping tablet: real 0.015s user 0.014s sys 0.000s
I20240501 19:56:56.860752 31311 raft_consensus.cc:348] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:56.860867 31311 raft_consensus.cc:729] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 474eb4491dcb47f4a2a2f6e149bc39c1, State: Initialized, Role: LEARNER
I20240501 19:56:56.860980 31311 consensus_queue.cc:260] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:56.861274 31311 ts_tablet_manager.cc:1398] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 19:56:56.861686 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-5a9674523a2148b1a6c1a5d22a57aac5 received from {username='slave'} at 127.20.210.8:56643
I20240501 19:56:56.861765 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-5a9674523a2148b1a6c1a5d22a57aac5 on tablet 5a9674523a2148b1a6c1a5d22a57aac5 with peer 474eb4491dcb47f4a2a2f6e149bc39c1
I20240501 19:56:56.872896 29008 raft_consensus.cc:1203] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:56.875109 31332 tablet_copy_client.cc:523] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:56.876164 31332 tablet_bootstrap.cc:492] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap starting.
I20240501 19:56:56.890543 31332 tablet_bootstrap.cc:492] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:56.890741 31332 tablet_bootstrap.cc:492] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap complete.
I20240501 19:56:56.890890 31332 ts_tablet_manager.cc:1367] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.005s
I20240501 19:56:56.891209 31332 raft_consensus.cc:348] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:56.891300 31332 raft_consensus.cc:729] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7ccb4fe66ab942d399551b10f5ed32ea, State: Initialized, Role: LEARNER
I20240501 19:56:56.891376 31332 consensus_queue.cc:260] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:56.891600 31332 ts_tablet_manager.cc:1398] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:56.892035 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-0aaa59df03844aadafb8d57797685dc2 received from {username='slave'} at 127.20.210.6:59627
I20240501 19:56:56.892107 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-0aaa59df03844aadafb8d57797685dc2 on tablet 0aaa59df03844aadafb8d57797685dc2 with peer 7ccb4fe66ab942d399551b10f5ed32ea
I20240501 19:56:56.893697 31358 tablet_copy_client.cc:523] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:56.894817 31358 tablet_bootstrap.cc:492] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap starting.
I20240501 19:56:56.910722 31358 tablet_bootstrap.cc:492] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:56.910954 31358 tablet_bootstrap.cc:492] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap complete.
I20240501 19:56:56.911109 31358 ts_tablet_manager.cc:1367] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.001s
I20240501 19:56:56.911478 31358 raft_consensus.cc:348] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:56.911567 31358 raft_consensus.cc:729] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 474eb4491dcb47f4a2a2f6e149bc39c1, State: Initialized, Role: LEARNER
I20240501 19:56:56.911643 31358 consensus_queue.cc:260] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:56.911888 31358 ts_tablet_manager.cc:1398] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:56:56.912312 27479 tablet_copy_service.cc:342] P e5057264da53436ea17bad01c66a7fbf: Request end of tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-6ee4db743833421e8120edbec4ba0836 received from {username='slave'} at 127.20.210.8:39999
I20240501 19:56:56.912398 27479 tablet_copy_service.cc:433] P e5057264da53436ea17bad01c66a7fbf: ending tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-6ee4db743833421e8120edbec4ba0836 on tablet 6ee4db743833421e8120edbec4ba0836 with peer 474eb4491dcb47f4a2a2f6e149bc39c1
I20240501 19:56:56.936430 29008 raft_consensus.cc:1203] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:56.945307 28877 raft_consensus.cc:1203] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:56.945549 29140 raft_consensus.cc:1203] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:56.976589 28877 raft_consensus.cc:1203] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:57.034111 29008 raft_consensus.cc:1203] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:57.065955 31439 tablet_copy_client.cc:523] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:57.067346 31439 tablet_bootstrap.cc:492] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap starting.
I20240501 19:56:57.068928 31444 tablet_copy_client.cc:523] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:56:57.070091 31444 tablet_bootstrap.cc:492] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap starting.
I20240501 19:56:57.091027 31439 tablet_bootstrap.cc:492] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:57.091266 31439 tablet_bootstrap.cc:492] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap complete.
I20240501 19:56:57.091416 31439 ts_tablet_manager.cc:1367] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent bootstrapping tablet: real 0.024s user 0.024s sys 0.000s
I20240501 19:56:57.091815 31439 raft_consensus.cc:348] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:57.091933 31439 raft_consensus.cc:729] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 474eb4491dcb47f4a2a2f6e149bc39c1, State: Initialized, Role: LEARNER
I20240501 19:56:57.092056 31439 consensus_queue.cc:260] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:57.092321 31439 ts_tablet_manager.cc:1398] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:56:57.092751 27085 tablet_copy_service.cc:342] P 16cfc5a5082742c5b09d35e5c4f7f10e: Request end of tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-c9f961f0554c45a1a3f31d5a8b31a46d received from {username='slave'} at 127.20.210.8:42289
I20240501 19:56:57.092832 27085 tablet_copy_service.cc:433] P 16cfc5a5082742c5b09d35e5c4f7f10e: ending tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-c9f961f0554c45a1a3f31d5a8b31a46d on tablet c9f961f0554c45a1a3f31d5a8b31a46d with peer 474eb4491dcb47f4a2a2f6e149bc39c1
I20240501 19:56:57.093457 31444 tablet_bootstrap.cc:492] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:56:57.093696 31444 tablet_bootstrap.cc:492] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap complete.
I20240501 19:56:57.093849 31444 ts_tablet_manager.cc:1367] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent bootstrapping tablet: real 0.024s user 0.021s sys 0.000s
I20240501 19:56:57.094213 31444 raft_consensus.cc:348] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:57.094329 31444 raft_consensus.cc:729] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 474eb4491dcb47f4a2a2f6e149bc39c1, State: Initialized, Role: LEARNER
I20240501 19:56:57.094416 31444 consensus_queue.cc:260] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:57.094655 31444 ts_tablet_manager.cc:1398] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 19:56:57.095086 27479 tablet_copy_service.cc:342] P e5057264da53436ea17bad01c66a7fbf: Request end of tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-3ca822a1462544c3be45371d6e088447 received from {username='slave'} at 127.20.210.8:39999
I20240501 19:56:57.095170 27479 tablet_copy_service.cc:433] P e5057264da53436ea17bad01c66a7fbf: ending tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-3ca822a1462544c3be45371d6e088447 on tablet 3ca822a1462544c3be45371d6e088447 with peer 474eb4491dcb47f4a2a2f6e149bc39c1
I20240501 19:56:57.184233 28746 raft_consensus.cc:1203] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:57.191025 31800 raft_consensus.cc:1050] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea to VOTER
I20240501 19:56:57.191419 31800 consensus_queue.cc:237] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:57.192593 27196 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.192608 27064 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.192694 28877 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.193037 31802 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.193222 31800 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.193384 30735 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.195395 30735 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.195871 27196 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.197489 27064 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.197557 28877 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.198478 26920 catalog_manager.cc:5484] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:57.202566 27328 consensus_queue.cc:237] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:57.203586 28877 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.203631 27064 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.204023 31802 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.204191 31800 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.205839 31802 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.206307 28877 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.206362 27064 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.207417 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3e05a10bfc6540c19faa154896b8976a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:57.208376 26915 catalog_manager.cc:5484] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea reported cstate change: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.210719 27175 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e05a10bfc6540c19faa154896b8976a with delete_type TABLET_DATA_TOMBSTONED (TS ef2d33d636574e8fbb9d3b90a15035d6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43438
I20240501 19:56:57.211004 31810 tablet_replica.cc:330] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6: stopping tablet replica
I20240501 19:56:57.211122 31810 raft_consensus.cc:2229] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:57.211267 31810 raft_consensus.cc:2258] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:57.211781 31810 ts_tablet_manager.cc:1875] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:57.213342 31810 ts_tablet_manager.cc:1888] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:57.213403 31810 log.cc:1196] T 3e05a10bfc6540c19faa154896b8976a P ef2d33d636574e8fbb9d3b90a15035d6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/3e05a10bfc6540c19faa154896b8976a
I20240501 19:56:57.213857 26908 catalog_manager.cc:4830] TS ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113): tablet 3e05a10bfc6540c19faa154896b8976a (table rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]) successfully deleted
I20240501 19:56:57.241840 29140 raft_consensus.cc:1203] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:57.330027 31917 raft_consensus.cc:1050] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: attempting to promote NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 to VOTER
I20240501 19:56:57.330433 31917 consensus_queue.cc:237] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:57.331605 27328 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.331624 29008 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.332012 31916 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.332325 31916 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.333489 27064 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.333890 31917 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.335971 30735 raft_consensus.cc:1050] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea to VOTER
I20240501 19:56:57.335871 31916 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.336365 30735 consensus_queue.cc:237] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:57.336344 27064 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.337585 27458 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.338008 31802 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.338171 28877 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.338951 31800 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.339114 29008 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.341500 26923 catalog_manager.cc:5484] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 reported cstate change: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.341641 27064 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.342051 31802 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.342496 31920 raft_consensus.cc:1050] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf: attempting to promote NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 to VOTER
I20240501 19:56:57.342974 31920 consensus_queue.cc:237] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:57.343981 27328 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.344182 27196 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.344391 29140 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.344591 31802 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.344743 31561 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.345068 31561 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.349457 27458 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.349668 31561 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.350107 28877 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.351433 27196 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.351948 26920 catalog_manager.cc:5484] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:57.353899 27328 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.354225 31931 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.354665 29139 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.354916 27064 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.355548 27328 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.357303 26924 catalog_manager.cc:5484] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.361164 27328 consensus_queue.cc:237] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:57.361248 27196 consensus_queue.cc:237] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:57.362375 27064 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.362425 29008 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.362607 27458 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.362725 31916 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.362895 31917 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.362943 31802 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.363097 28877 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.363890 31802 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.364600 31916 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.365000 27064 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.365250 29008 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.365489 31802 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.366360 26908 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 55d39ed3702046b08b1ac3c76c272c01 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:57.366909 27458 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.367669 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0aaa59df03844aadafb8d57797685dc2 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:57.367599 26922 catalog_manager.cc:5484] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 reported cstate change: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.369572 28877 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.369805 26916 catalog_manager.cc:5484] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:57.370738 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d39ed3702046b08b1ac3c76c272c01 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3f70fdad7b438a841f74fc638d17f5 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34348
I20240501 19:56:57.372002 31934 tablet_replica.cc:330] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5: stopping tablet replica
I20240501 19:56:57.372217 31934 raft_consensus.cc:2229] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:57.372272 27045 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aaa59df03844aadafb8d57797685dc2 with delete_type TABLET_DATA_TOMBSTONED (TS 16cfc5a5082742c5b09d35e5c4f7f10e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49644
I20240501 19:56:57.372447 31934 raft_consensus.cc:2258] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:57.372611 31935 tablet_replica.cc:330] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e: stopping tablet replica
I20240501 19:56:57.372776 31935 raft_consensus.cc:2229] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:57.372946 31935 raft_consensus.cc:2258] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:57.373147 31934 ts_tablet_manager.cc:1875] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:57.374037 31935 ts_tablet_manager.cc:1875] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:57.375205 31934 ts_tablet_manager.cc:1888] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:57.375294 31934 log.cc:1196] T 55d39ed3702046b08b1ac3c76c272c01 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/55d39ed3702046b08b1ac3c76c272c01
I20240501 19:56:57.375777 26910 catalog_manager.cc:4830] TS 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155): tablet 55d39ed3702046b08b1ac3c76c272c01 (table rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]) successfully deleted
I20240501 19:56:57.376122 31935 ts_tablet_manager.cc:1888] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:57.376231 31935 log.cc:1196] T 0aaa59df03844aadafb8d57797685dc2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/0aaa59df03844aadafb8d57797685dc2
I20240501 19:56:57.376775 26909 catalog_manager.cc:4830] TS 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375): tablet 0aaa59df03844aadafb8d57797685dc2 (table rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]) successfully deleted
I20240501 19:56:57.404435 29140 raft_consensus.cc:1203] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:57.422667 29140 raft_consensus.cc:1203] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:56:57.449328 31800 raft_consensus.cc:1050] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea to VOTER
I20240501 19:56:57.449702 31800 consensus_queue.cc:237] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:57.451078 27064 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.451174 27458 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.451184 28877 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.451509 31438 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.451697 31800 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.451879 30735 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.453823 31800 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.454144 28877 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.454317 27064 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.456871 26916 catalog_manager.cc:5484] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.457305 27458 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.460106 27328 consensus_queue.cc:237] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:56:57.461196 27458 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.461241 28877 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.461549 31800 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.461853 31800 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.463096 31802 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.463801 28877 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.464337 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0723850372f8416d9b8ac19f119939bc with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:57.464494 27458 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:56:57.465770 26916 catalog_manager.cc:5484] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 5 to 6, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:57.467888 27045 tablet_service.cc:1514] Processing DeleteTablet for tablet 0723850372f8416d9b8ac19f119939bc with delete_type TABLET_DATA_TOMBSTONED (TS 16cfc5a5082742c5b09d35e5c4f7f10e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49644
I20240501 19:56:57.468111 31935 tablet_replica.cc:330] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e: stopping tablet replica
I20240501 19:56:57.468250 31935 raft_consensus.cc:2229] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:57.468410 31935 raft_consensus.cc:2258] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:57.468991 31935 ts_tablet_manager.cc:1875] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:57.470661 31935 ts_tablet_manager.cc:1888] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:57.470724 31935 log.cc:1196] T 0723850372f8416d9b8ac19f119939bc P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/0723850372f8416d9b8ac19f119939bc
I20240501 19:56:57.471170 26909 catalog_manager.cc:4830] TS 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375): tablet 0723850372f8416d9b8ac19f119939bc (table rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]) successfully deleted
I20240501 19:56:57.478624 31618 raft_consensus.cc:1050] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6: attempting to promote NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 to VOTER
I20240501 19:56:57.479044 31618 consensus_queue.cc:237] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:57.480244 27064 raft_consensus.cc:1261] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.480280 27328 raft_consensus.cc:1261] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.480283 29008 raft_consensus.cc:1261] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.480659 31941 consensus_queue.cc:1035] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.480824 31916 consensus_queue.cc:1035] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.481050 31618 consensus_queue.cc:1035] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.482686 31916 raft_consensus.cc:2932] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.483307 29008 raft_consensus.cc:2932] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.483453 27064 raft_consensus.cc:2932] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.484959 26924 catalog_manager.cc:5484] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:57.485260 27328 raft_consensus.cc:2932] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.565295 27196 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b3e84419a13348e4a0e7bacfe94cf33e"
dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:41556
I20240501 19:56:57.565472 27196 raft_consensus.cc:595] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:56:57.606631 27196 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b3e84419a13348e4a0e7bacfe94cf33e"
dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:41590
I20240501 19:56:57.606760 27196 raft_consensus.cc:595] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:56:57.652927 31438 raft_consensus.cc:1050] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 to VOTER
I20240501 19:56:57.653365 31438 consensus_queue.cc:237] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:57.654785 29008 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.654834 27458 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.655025 27064 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.655155 31800 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.655450 31802 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.655622 31438 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.657589 31438 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.658901 27064 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.660645 29008 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.661146 26922 catalog_manager.cc:5484] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.663246 27458 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:57.748656 32045 raft_consensus.cc:1050] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e: attempting to promote NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 to VOTER
I20240501 19:56:57.749130 32045 consensus_queue.cc:237] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:57.750406 27196 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.750453 29140 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.750726 27328 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.750810 32044 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.751080 32045 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.751377 32045 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.753041 32044 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.753468 27196 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.754030 29140 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.755631 26924 catalog_manager.cc:5484] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:57.756089 27328 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.758975 27064 consensus_queue.cc:237] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:57.760140 27196 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.760219 29140 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.760593 31562 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.761121 31562 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.762661 32045 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.763002 29140 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.763125 27196 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.763830 26909 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c9f961f0554c45a1a3f31d5a8b31a46d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:57.764667 26922 catalog_manager.cc:5484] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 5 to 6, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:57.766813 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet c9f961f0554c45a1a3f31d5a8b31a46d with delete_type TABLET_DATA_TOMBSTONED (TS 3c3f70fdad7b438a841f74fc638d17f5 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34348
I20240501 19:56:57.767017 31934 tablet_replica.cc:330] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5: stopping tablet replica
I20240501 19:56:57.767153 31934 raft_consensus.cc:2229] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:57.767289 31934 raft_consensus.cc:2258] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:57.767918 31934 ts_tablet_manager.cc:1875] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:57.769505 31934 ts_tablet_manager.cc:1888] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:57.769563 31934 log.cc:1196] T c9f961f0554c45a1a3f31d5a8b31a46d P 3c3f70fdad7b438a841f74fc638d17f5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/c9f961f0554c45a1a3f31d5a8b31a46d
I20240501 19:56:57.769948 26910 catalog_manager.cc:4830] TS 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155): tablet c9f961f0554c45a1a3f31d5a8b31a46d (table rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]) successfully deleted
I20240501 19:56:57.783210 30735 raft_consensus.cc:1050] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER 724bae28280242c6982c4f8e315b0d0b to VOTER
I20240501 19:56:57.783564 30735 consensus_queue.cc:237] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:56:57.784539 27458 consensus_queue.cc:237] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:57.785104 27459 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.785262 27196 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.785490 31438 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.785586 28746 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.785734 31800 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.785952 29140 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.786057 31800 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.786154 27328 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.786500 31931 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.786680 31932 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.787961 31800 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:57.788342 28746 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:57.788406 31931 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.788652 27196 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:57.789845 26911 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6ee4db743833421e8120edbec4ba0836 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:57.790259 27459 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:57.790989 26916 catalog_manager.cc:5484] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 4 to 5, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:57.791373 26915 catalog_manager.cc:5484] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:57.791823 27328 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.792147 29140 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.794600 27175 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ee4db743833421e8120edbec4ba0836 with delete_type TABLET_DATA_TOMBSTONED (TS ef2d33d636574e8fbb9d3b90a15035d6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43438
I20240501 19:56:57.795183 27328 consensus_queue.cc:237] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:56:57.795257 32057 tablet_replica.cc:330] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6: stopping tablet replica
I20240501 19:56:57.795594 32057 raft_consensus.cc:2229] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:57.795809 32057 raft_consensus.cc:2258] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:57.796375 27459 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.796715 32057 ts_tablet_manager.cc:1875] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:57.796900 28746 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.796932 30735 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.797312 31438 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.798403 31438 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:57.799225 32057 ts_tablet_manager.cc:1888] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:57.799219 27459 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:57.799345 32057 log.cc:1196] T 6ee4db743833421e8120edbec4ba0836 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/6ee4db743833421e8120edbec4ba0836
I20240501 19:56:57.799216 28746 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:56:57.799919 26908 catalog_manager.cc:4830] TS ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113): tablet 6ee4db743833421e8120edbec4ba0836 (table rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]) successfully deleted
I20240501 19:56:57.799975 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 75cc96baf4654318a09dcc34eb32819c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:57.801316 26924 catalog_manager.cc:5484] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:57.803236 27175 tablet_service.cc:1514] Processing DeleteTablet for tablet 75cc96baf4654318a09dcc34eb32819c with delete_type TABLET_DATA_TOMBSTONED (TS ef2d33d636574e8fbb9d3b90a15035d6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43438
I20240501 19:56:57.803436 32057 tablet_replica.cc:330] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6: stopping tablet replica
I20240501 19:56:57.803571 32057 raft_consensus.cc:2229] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:57.803705 32057 raft_consensus.cc:2258] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:57.804205 32057 ts_tablet_manager.cc:1875] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:57.805624 32057 ts_tablet_manager.cc:1888] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:57.805683 32057 log.cc:1196] T 75cc96baf4654318a09dcc34eb32819c P ef2d33d636574e8fbb9d3b90a15035d6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/75cc96baf4654318a09dcc34eb32819c
I20240501 19:56:57.806087 26908 catalog_manager.cc:4830] TS ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113): tablet 75cc96baf4654318a09dcc34eb32819c (table rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]) successfully deleted
I20240501 19:56:57.840755 27328 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1"
dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:36566
I20240501 19:56:57.840864 27328 raft_consensus.cc:595] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:56:57.850103 27196 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b3e84419a13348e4a0e7bacfe94cf33e"
dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:41616
I20240501 19:56:57.850221 27196 raft_consensus.cc:595] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:56:57.873524 27328 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1"
dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:36600
I20240501 19:56:57.873641 27328 raft_consensus.cc:595] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:56:57.882966 27196 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b3e84419a13348e4a0e7bacfe94cf33e"
dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:41636
I20240501 19:56:57.883090 27196 raft_consensus.cc:595] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:56:57.899407 31922 raft_consensus.cc:1050] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf: attempting to promote NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 to VOTER
I20240501 19:56:57.899755 31922 consensus_queue.cc:237] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:57.901135 27064 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.901135 27196 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.901202 29140 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:57.901495 31561 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.901706 31932 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.901976 31920 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:57.903940 31922 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.905903 29140 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.905987 27196 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.907349 27064 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.909168 26922 catalog_manager.cc:5484] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.912312 27459 consensus_queue.cc:237] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:57.913281 29140 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.913488 27064 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:57.913611 31922 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.914168 31922 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:57.915163 31932 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.915540 29140 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.916191 27064 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.916607 26911 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3ca822a1462544c3be45371d6e088447 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:57.918380 26916 catalog_manager.cc:5484] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: config changed from index 5 to 6, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:57.918560 31941 raft_consensus.cc:979] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6: : Instructing follower 16cfc5a5082742c5b09d35e5c4f7f10e to start an election
I20240501 19:56:57.918648 31917 raft_consensus.cc:1067] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Signalling peer 16cfc5a5082742c5b09d35e5c4f7f10e to start an election
I20240501 19:56:57.919157 27064 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b3e84419a13348e4a0e7bacfe94cf33e"
dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
from {username='slave'} at 127.20.210.2:36499
I20240501 19:56:57.919286 27064 raft_consensus.cc:482] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 19:56:57.919344 27064 raft_consensus.cc:3037] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:56:57.920758 27175 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ca822a1462544c3be45371d6e088447 with delete_type TABLET_DATA_TOMBSTONED (TS ef2d33d636574e8fbb9d3b90a15035d6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43438
I20240501 19:56:57.921178 32057 tablet_replica.cc:330] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6: stopping tablet replica
I20240501 19:56:57.921319 32057 raft_consensus.cc:2229] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:57.921458 32057 raft_consensus.cc:2258] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:57.920048 27064 raft_consensus.cc:504] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:57.922024 27064 leader_election.cc:288] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 2 election: Requested vote from peers 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695)
I20240501 19:56:57.922340 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3e84419a13348e4a0e7bacfe94cf33e" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:57.922456 27328 raft_consensus.cc:3037] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:56:57.922552 29008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3e84419a13348e4a0e7bacfe94cf33e" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20"
I20240501 19:56:57.922731 32057 ts_tablet_manager.cc:1875] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:57.922737 29008 raft_consensus.cc:3037] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:56:57.923470 27328 raft_consensus.cc:2454] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 2.
I20240501 19:56:57.923820 29008 raft_consensus.cc:2454] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 2.
I20240501 19:56:57.924142 27001 leader_election.cc:302] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, 3c3f70fdad7b438a841f74fc638d17f5, ecdc0d100f5f4b95b9dda10c2f97cf20; no voters:
I20240501 19:56:57.924147 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3e84419a13348e4a0e7bacfe94cf33e" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:56:57.924247 27196 raft_consensus.cc:3032] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 19:56:57.924284 27196 raft_consensus.cc:729] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Running, Role: LEADER
I20240501 19:56:57.924456 27196 consensus_queue.cc:260] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:57.924628 27196 raft_consensus.cc:3037] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:56:57.924782 32057 ts_tablet_manager.cc:1888] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:57.924844 32057 log.cc:1196] T 3ca822a1462544c3be45371d6e088447 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/3ca822a1462544c3be45371d6e088447
I20240501 19:56:57.925236 31562 raft_consensus.cc:2781] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Leader election won for term 2
I20240501 19:56:57.925278 26908 catalog_manager.cc:4830] TS ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113): tablet 3ca822a1462544c3be45371d6e088447 (table rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]) successfully deleted
I20240501 19:56:57.925328 31562 raft_consensus.cc:686] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 LEADER]: Becoming Leader. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Running, Role: LEADER
I20240501 19:56:57.925364 27196 raft_consensus.cc:2454] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 2.
I20240501 19:56:57.925467 31562 consensus_queue.cc:237] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:57.926954 26916 catalog_manager.cc:5484] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: term changed from 1 to 2, leader changed from ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) to 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1). New cstate: current_term: 2 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:58.001669 31800 raft_consensus.cc:1050] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 to VOTER
I20240501 19:56:58.002032 31800 consensus_queue.cc:237] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:58.003312 27459 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:58.003355 29140 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:58.003369 27196 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:56:58.003697 30735 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:58.003875 31800 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:58.004046 31438 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:56:58.006052 31800 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:58.006476 27459 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:58.007656 27196 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:58.007891 29140 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:58.008620 26916 catalog_manager.cc:5484] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 4 to 5, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:58.011664 27328 consensus_queue.cc:237] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:56:58.012753 27196 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:58.012771 29140 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:56:58.013099 31802 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:58.013242 31438 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:58.014591 31802 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:58.015038 27196 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:58.015038 29140 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:56:58.016363 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5a9674523a2148b1a6c1a5d22a57aac5 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:58.016870 26922 catalog_manager.cc:5484] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 5 to 6, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:56:58.018923 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a9674523a2148b1a6c1a5d22a57aac5 with delete_type TABLET_DATA_TOMBSTONED (TS e5057264da53436ea17bad01c66a7fbf not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:41962
I20240501 19:56:58.019256 32141 tablet_replica.cc:330] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf: stopping tablet replica
I20240501 19:56:58.019380 32141 raft_consensus.cc:2229] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:58.019745 32141 raft_consensus.cc:2258] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:58.020327 32141 ts_tablet_manager.cc:1875] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:58.021847 32141 ts_tablet_manager.cc:1888] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:56:58.021929 32141 log.cc:1196] T 5a9674523a2148b1a6c1a5d22a57aac5 P e5057264da53436ea17bad01c66a7fbf: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/5a9674523a2148b1a6c1a5d22a57aac5
I20240501 19:56:58.022377 26911 catalog_manager.cc:4830] TS e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115): tablet 5a9674523a2148b1a6c1a5d22a57aac5 (table rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]) successfully deleted
I20240501 19:56:58.048923 31802 raft_consensus.cc:979] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5: : Instructing follower 16cfc5a5082742c5b09d35e5c4f7f10e to start an election
I20240501 19:56:58.049021 31802 raft_consensus.cc:1067] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Signalling peer 16cfc5a5082742c5b09d35e5c4f7f10e to start an election
I20240501 19:56:58.049420 27064 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1"
dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
from {username='slave'} at 127.20.210.3:39617
I20240501 19:56:58.049540 27064 raft_consensus.cc:482] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 19:56:58.049585 27064 raft_consensus.cc:3037] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:56:58.050359 27064 raft_consensus.cc:504] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:58.050760 27064 leader_election.cc:288] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 2 election: Requested vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695)
I20240501 19:56:58.051015 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:56:58.051044 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:56:58.051115 27328 raft_consensus.cc:3032] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 19:56:58.051143 27459 raft_consensus.cc:3037] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:56:58.051156 27328 raft_consensus.cc:729] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:56:58.051163 29008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1" candidate_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20"
I20240501 19:56:58.051308 29008 raft_consensus.cc:3037] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:56:58.051304 27328 consensus_queue.cc:260] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:58.051468 27328 raft_consensus.cc:3037] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:56:58.052032 27459 raft_consensus.cc:2454] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 2.
I20240501 19:56:58.052217 29008 raft_consensus.cc:2454] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 2.
I20240501 19:56:58.052403 27328 raft_consensus.cc:2454] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16cfc5a5082742c5b09d35e5c4f7f10e in term 2.
I20240501 19:56:58.052513 27001 leader_election.cc:302] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, e5057264da53436ea17bad01c66a7fbf, ecdc0d100f5f4b95b9dda10c2f97cf20; no voters:
I20240501 19:56:58.052732 31562 raft_consensus.cc:2781] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Leader election won for term 2
I20240501 19:56:58.052824 31562 raft_consensus.cc:686] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 LEADER]: Becoming Leader. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Running, Role: LEADER
I20240501 19:56:58.052965 31562 consensus_queue.cc:237] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:58.054507 26924 catalog_manager.cc:5484] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: term changed from 1 to 2, leader changed from 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) to 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1). New cstate: current_term: 2 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:58.419245 29008 raft_consensus.cc:1261] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 19:56:58.419647 31562 consensus_queue.cc:1035] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:58.475920 29008 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 19:56:58.476294 31562 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:58.510043 27328 raft_consensus.cc:1261] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 19:56:58.510421 31562 consensus_queue.cc:1035] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:58.513088 27196 raft_consensus.cc:1261] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 19:56:58.514422 32047 consensus_queue.cc:1035] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:58.517068 27064 consensus_queue.cc:237] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:58.518260 29008 raft_consensus.cc:1261] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 19:56:58.518606 27328 raft_consensus.cc:1261] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 19:56:58.518811 32187 consensus_queue.cc:1035] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:58.518970 32184 consensus_queue.cc:1035] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:58.520411 32047 raft_consensus.cc:2932] T b3e84419a13348e4a0e7bacfe94cf33e P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:58.521008 27328 raft_consensus.cc:2932] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:58.520998 29008 raft_consensus.cc:2932] T b3e84419a13348e4a0e7bacfe94cf33e P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:58.522209 26909 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b3e84419a13348e4a0e7bacfe94cf33e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:58.523375 26915 catalog_manager.cc:5484] T b3e84419a13348e4a0e7bacfe94cf33e P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 5 to 7, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New cstate: current_term: 2 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:58.525832 27175 tablet_service.cc:1514] Processing DeleteTablet for tablet b3e84419a13348e4a0e7bacfe94cf33e with delete_type TABLET_DATA_TOMBSTONED (TS ef2d33d636574e8fbb9d3b90a15035d6 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:43438
I20240501 19:56:58.526100 32190 tablet_replica.cc:330] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6: stopping tablet replica
I20240501 19:56:58.526283 32190 raft_consensus.cc:2229] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:58.526387 32190 raft_consensus.cc:2258] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:58.527011 32190 ts_tablet_manager.cc:1875] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:58.528816 32190 ts_tablet_manager.cc:1888] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240501 19:56:58.528879 32190 log.cc:1196] T b3e84419a13348e4a0e7bacfe94cf33e P ef2d33d636574e8fbb9d3b90a15035d6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/b3e84419a13348e4a0e7bacfe94cf33e
I20240501 19:56:58.529312 26908 catalog_manager.cc:4830] TS ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113): tablet b3e84419a13348e4a0e7bacfe94cf33e (table rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]) successfully deleted
I20240501 19:56:58.614467 27328 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 19:56:58.614938 32184 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:58.617661 27459 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 19:56:58.618976 32187 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:56:59.032605 27064 consensus_queue.cc:237] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:56:59.033702 27459 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 19:56:59.033928 29008 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 19:56:59.034063 32184 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.034309 32187 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.035826 31562 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:59.036291 27459 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:59.036298 29008 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:59.037851 26909 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 692732b291b441029826cf20c9a0ecb1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:56:59.038808 26924 catalog_manager.cc:5484] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 5 to 7, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New cstate: current_term: 2 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:56:59.041452 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet 692732b291b441029826cf20c9a0ecb1 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3f70fdad7b438a841f74fc638d17f5 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:34348
I20240501 19:56:59.041790 32221 tablet_replica.cc:330] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5: stopping tablet replica
I20240501 19:56:59.041903 32221 raft_consensus.cc:2229] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 19:56:59.041981 32221 raft_consensus.cc:2258] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 19:56:59.042518 32221 ts_tablet_manager.cc:1875] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:56:59.044338 32221 ts_tablet_manager.cc:1888] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240501 19:56:59.044401 32221 log.cc:1196] T 692732b291b441029826cf20c9a0ecb1 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/692732b291b441029826cf20c9a0ecb1
I20240501 19:56:59.044813 26910 catalog_manager.cc:4830] TS 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155): tablet 692732b291b441029826cf20c9a0ecb1 (table rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]) successfully deleted
I20240501 19:56:59.231654 27045 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 19:56:59.236166 27307 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240501 19:56:59.239835 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:59.241375 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:59.241847 29120 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:59.251569 27437 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:56:59.252755 27175 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:56:59.255719 30019 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:56:59.275408 28988 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:56:59.311524 27064 consensus_queue.cc:237] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:56:59.312896 27459 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.312925 28876 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.313395 32184 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.313802 32184 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.315467 32047 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.317719 27459 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.320423 26918 catalog_manager.cc:5484] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.320428 28876 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
W20240501 19:56:59.322081 27002 consensus_peers.cc:485] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e -> Peer 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833): Couldn't send request to peer 045b41d2fb5545fd9975510efab305aa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37288cafed764ba18fcc07b6e3c7a722. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.330016 27328 consensus_queue.cc:237] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:59.331393 27064 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.331401 29008 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.331784 32290 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.331949 31802 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.333557 32290 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.334019 29008 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
W20240501 19:56:59.334729 27263 consensus_peers.cc:485] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer 724bae28280242c6982c4f8e315b0d0b (127.20.210.5:42343): Couldn't send request to peer 724bae28280242c6982c4f8e315b0d0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f149fd8790a0412e81ac49d0ce359ede. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.336681 26916 catalog_manager.cc:5484] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:59.343843 27328 consensus_queue.cc:237] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:56:59.344379 27064 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
W20240501 19:56:59.345280 27263 consensus_peers.cc:485] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8:44887): Couldn't send request to peer 474eb4491dcb47f4a2a2f6e149bc39c1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0aaa59df03844aadafb8d57797685dc2. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.346311 28876 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.346639 31802 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.346974 27459 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.347324 31802 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.348634 32291 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:59.349004 28876 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:59.349838 27459 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:59.351243 26918 catalog_manager.cc:5484] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea reported cstate change: config changed from index 6 to 7, NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } } }
I20240501 19:56:59.360003 27064 consensus_queue.cc:237] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:56:59.361250 29140 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.361258 27196 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 19:56:59.361603 27002 consensus_peers.cc:485] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e -> Peer 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833): Couldn't send request to peer 045b41d2fb5545fd9975510efab305aa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c9f961f0554c45a1a3f31d5a8b31a46d. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.361608 32047 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.361840 32187 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.363495 32184 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.364236 27196 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.365136 29140 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.366874 26915 catalog_manager.cc:5484] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:59.374732 27459 consensus_queue.cc:237] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:59.376343 29140 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.376359 27064 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.376729 32317 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.376966 32319 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.379372 32319 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
W20240501 19:56:59.379815 27394 consensus_peers.cc:485] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf -> Peer ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695): Couldn't send request to peer ecdc0d100f5f4b95b9dda10c2f97cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ca822a1462544c3be45371d6e088447. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.379772 29140 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:59.381114 27064 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:59.381880 26916 catalog_manager.cc:5484] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 6 to 7, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:59.390851 27196 consensus_queue.cc:237] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:56:59.392166 27459 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.392233 28746 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 19:56:59.392581 27131 consensus_peers.cc:485] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 -> Peer 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529): Couldn't send request to peer 7ccb4fe66ab942d399551b10f5ed32ea. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0f417f4001bc48158187ed262f7737ab. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.392609 31941 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.392792 32330 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.394438 32329 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:59.394891 28746 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:59.394958 27459 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:59.397090 26915 catalog_manager.cc:5484] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 6 to 7, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:56:59.405182 27064 consensus_queue.cc:237] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:59.406561 29008 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.406684 27196 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.406939 32184 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.407104 32187 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.409183 32047 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.409925 29008 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
W20240501 19:56:59.410063 27001 consensus_peers.cc:485] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e -> Peer 724bae28280242c6982c4f8e315b0d0b (127.20.210.5:42343): Couldn't send request to peer 724bae28280242c6982c4f8e315b0d0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c498eacc5e947bfb01dfb90c7039df0. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.410027 27196 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.412467 26916 catalog_manager.cc:5484] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 reported cstate change: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.419253 27328 consensus_queue.cc:237] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:56:59.420572 27196 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.420702 29140 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.420876 31802 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.421077 32290 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.422834 32291 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.423295 29140 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
W20240501 19:56:59.424898 27264 consensus_peers.cc:485] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833): Couldn't send request to peer 045b41d2fb5545fd9975510efab305aa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5a9674523a2148b1a6c1a5d22a57aac5. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.425652 26918 catalog_manager.cc:5484] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:59.426535 27196 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.433455 27196 consensus_queue.cc:237] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:56:59.434804 27064 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.434816 29008 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.435200 32329 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.435559 32329 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.436956 31941 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.437397 27064 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.438184 29008 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.439993 26924 catalog_manager.cc:5484] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 19:56:59.445276 27133 consensus_peers.cc:485] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 -> Peer 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833): Couldn't send request to peer 045b41d2fb5545fd9975510efab305aa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 55d39ed3702046b08b1ac3c76c272c01. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.452224 27328 consensus_queue.cc:237] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:56:59.453452 27064 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 19:56:59.453536 27263 consensus_peers.cc:485] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695): Couldn't send request to peer ecdc0d100f5f4b95b9dda10c2f97cf20. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e05a10bfc6540c19faa154896b8976a. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.453572 28876 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.453910 31802 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.454087 32290 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.455778 32291 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:59.456223 27064 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:59.456264 28876 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } } }
I20240501 19:56:59.458560 26918 catalog_manager.cc:5484] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 6 to 7, NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:59.465899 27196 consensus_queue.cc:237] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:59.467298 29140 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.467384 27459 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 19:56:59.467453 27132 consensus_peers.cc:485] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 -> Peer 724bae28280242c6982c4f8e315b0d0b (127.20.210.5:42343): Couldn't send request to peer 724bae28280242c6982c4f8e315b0d0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 896072a285b649dbb31f9f02666ace25. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.467650 31941 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.467813 32330 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.469540 32329 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.470011 29140 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.470110 27459 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.472180 26915 catalog_manager.cc:5484] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:56:59.479184 27459 consensus_queue.cc:237] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:56:59.480446 29140 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.480564 27328 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.480890 32207 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.481112 32318 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.482650 32207 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.483184 29140 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.484447 27328 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:56:59.485534 26924 catalog_manager.cc:5484] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
W20240501 19:56:59.486166 27395 consensus_peers.cc:485] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf -> Peer 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833): Couldn't send request to peer 045b41d2fb5545fd9975510efab305aa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9102f7c8350843c2bd7009ea190860d0. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.502605 27064 consensus_queue.cc:237] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:59.503976 27459 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.504022 28876 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 19:56:59.504238 27001 consensus_peers.cc:485] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e -> Peer 724bae28280242c6982c4f8e315b0d0b (127.20.210.5:42343): Couldn't send request to peer 724bae28280242c6982c4f8e315b0d0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 199f9529870242f599358995c19250b2. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.504390 31562 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.504570 32184 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.506301 32184 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.506614 27459 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.508589 28876 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.509157 26915 catalog_manager.cc:5484] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.517714 27196 consensus_queue.cc:237] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:56:59.519034 28876 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:56:59.519111 27459 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 19:56:59.519258 27132 consensus_peers.cc:485] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 -> Peer 724bae28280242c6982c4f8e315b0d0b (127.20.210.5:42343): Couldn't send request to peer 724bae28280242c6982c4f8e315b0d0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2c944ca787b475294f9c41f6052a7ea. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:56:59.519444 31941 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.519616 32329 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:56:59.521096 32330 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.521462 27459 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.522958 28876 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.523550 26924 catalog_manager.cc:5484] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 6 to 7, NON_VOTER 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) added. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } } }
I20240501 19:56:59.743583 32470 ts_tablet_manager.cc:902] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:59.744105 32470 tablet_copy_client.cc:308] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:59.744551 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet f149fd8790a0412e81ac49d0ce359ede from peer 724bae28280242c6982c4f8e315b0d0b ({username='slave'} at 127.20.210.5:40583)
I20240501 19:56:59.744644 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet f149fd8790a0412e81ac49d0ce359ede from peer 724bae28280242c6982c4f8e315b0d0b at {username='slave'} at 127.20.210.5:40583: session id = 724bae28280242c6982c4f8e315b0d0b-f149fd8790a0412e81ac49d0ce359ede
I20240501 19:56:59.745311 27348 tablet_copy_source_session.cc:205] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:59.746069 32470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f149fd8790a0412e81ac49d0ce359ede. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:59.747865 32470 tablet_copy_client.cc:783] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:59.748003 32470 tablet_copy_client.cc:647] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:59.748965 32474 tablet_copy_client.cc:1002] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:59.764818 32475 ts_tablet_manager.cc:902] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20: Initiating tablet copy from peer e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:56:59.765233 32475 tablet_copy_client.cc:308] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:37115
I20240501 19:56:59.768584 27479 tablet_copy_service.cc:140] P e5057264da53436ea17bad01c66a7fbf: Received BeginTabletCopySession request for tablet 3ca822a1462544c3be45371d6e088447 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 ({username='slave'} at 127.20.210.7:47243)
I20240501 19:56:59.768671 27479 tablet_copy_service.cc:161] P e5057264da53436ea17bad01c66a7fbf: Beginning new tablet copy session on tablet 3ca822a1462544c3be45371d6e088447 from peer ecdc0d100f5f4b95b9dda10c2f97cf20 at {username='slave'} at 127.20.210.7:47243: session id = ecdc0d100f5f4b95b9dda10c2f97cf20-3ca822a1462544c3be45371d6e088447
I20240501 19:56:59.769452 27479 tablet_copy_source_session.cc:205] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:59.770217 32475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ca822a1462544c3be45371d6e088447. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:59.771783 32475 tablet_copy_client.cc:783] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:59.771922 32475 tablet_copy_client.cc:647] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:59.772854 32479 tablet_copy_client.cc:1002] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:59.786844 32481 ts_tablet_manager.cc:902] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b: Initiating tablet copy from peer 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:59.787276 32481 tablet_copy_client.cc:308] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:33375
I20240501 19:56:59.790141 27085 tablet_copy_service.cc:140] P 16cfc5a5082742c5b09d35e5c4f7f10e: Received BeginTabletCopySession request for tablet 6c498eacc5e947bfb01dfb90c7039df0 from peer 724bae28280242c6982c4f8e315b0d0b ({username='slave'} at 127.20.210.5:49029)
I20240501 19:56:59.790225 27085 tablet_copy_service.cc:161] P 16cfc5a5082742c5b09d35e5c4f7f10e: Beginning new tablet copy session on tablet 6c498eacc5e947bfb01dfb90c7039df0 from peer 724bae28280242c6982c4f8e315b0d0b at {username='slave'} at 127.20.210.5:49029: session id = 724bae28280242c6982c4f8e315b0d0b-6c498eacc5e947bfb01dfb90c7039df0
I20240501 19:56:59.790948 27085 tablet_copy_source_session.cc:205] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:59.791700 32481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c498eacc5e947bfb01dfb90c7039df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:59.793609 32481 tablet_copy_client.cc:783] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:59.793738 32481 tablet_copy_client.cc:647] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:59.794616 32485 tablet_copy_client.cc:1002] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:59.804947 32502 ts_tablet_manager.cc:902] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:59.805447 32502 tablet_copy_client.cc:308] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:59.805873 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 0aaa59df03844aadafb8d57797685dc2 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 ({username='slave'} at 127.20.210.8:56643)
I20240501 19:56:59.805976 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 0aaa59df03844aadafb8d57797685dc2 from peer 474eb4491dcb47f4a2a2f6e149bc39c1 at {username='slave'} at 127.20.210.8:56643: session id = 474eb4491dcb47f4a2a2f6e149bc39c1-0aaa59df03844aadafb8d57797685dc2
I20240501 19:56:59.806924 27348 tablet_copy_source_session.cc:205] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:59.807229 32507 ts_tablet_manager.cc:902] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa: Initiating tablet copy from peer 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:59.807766 32502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aaa59df03844aadafb8d57797685dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:59.809739 32502 tablet_copy_client.cc:783] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:59.809849 32502 tablet_copy_client.cc:647] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:59.810724 32507 tablet_copy_client.cc:308] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:33375
I20240501 19:56:59.810938 32506 tablet_copy_client.cc:1002] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:59.814146 27085 tablet_copy_service.cc:140] P 16cfc5a5082742c5b09d35e5c4f7f10e: Received BeginTabletCopySession request for tablet c9f961f0554c45a1a3f31d5a8b31a46d from peer 045b41d2fb5545fd9975510efab305aa ({username='slave'} at 127.20.210.9:56871)
I20240501 19:56:59.814222 27085 tablet_copy_service.cc:161] P 16cfc5a5082742c5b09d35e5c4f7f10e: Beginning new tablet copy session on tablet c9f961f0554c45a1a3f31d5a8b31a46d from peer 045b41d2fb5545fd9975510efab305aa at {username='slave'} at 127.20.210.9:56871: session id = 045b41d2fb5545fd9975510efab305aa-c9f961f0554c45a1a3f31d5a8b31a46d
I20240501 19:56:59.814793 27085 tablet_copy_source_session.cc:205] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:59.817430 32507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9f961f0554c45a1a3f31d5a8b31a46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:59.821731 32523 ts_tablet_manager.cc:902] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:56:59.822119 32523 tablet_copy_client.cc:308] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:56:59.822723 32507 tablet_copy_client.cc:783] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:59.822919 32507 tablet_copy_client.cc:647] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:59.824231 32514 tablet_copy_client.cc:1002] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:59.826045 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 5a9674523a2148b1a6c1a5d22a57aac5 from peer 045b41d2fb5545fd9975510efab305aa ({username='slave'} at 127.20.210.9:37755)
I20240501 19:56:59.826120 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 5a9674523a2148b1a6c1a5d22a57aac5 from peer 045b41d2fb5545fd9975510efab305aa at {username='slave'} at 127.20.210.9:37755: session id = 045b41d2fb5545fd9975510efab305aa-5a9674523a2148b1a6c1a5d22a57aac5
I20240501 19:56:59.827049 27348 tablet_copy_source_session.cc:205] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:59.828063 32523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a9674523a2148b1a6c1a5d22a57aac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:59.830044 32523 tablet_copy_client.cc:783] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:59.830281 32523 tablet_copy_client.cc:647] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:59.831279 32525 tablet_copy_client.cc:1002] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:59.847349 32554 ts_tablet_manager.cc:902] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa: Initiating tablet copy from peer 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:56:59.847944 32554 tablet_copy_client.cc:308] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:33375
I20240501 19:56:59.848392 27085 tablet_copy_service.cc:140] P 16cfc5a5082742c5b09d35e5c4f7f10e: Received BeginTabletCopySession request for tablet 37288cafed764ba18fcc07b6e3c7a722 from peer 045b41d2fb5545fd9975510efab305aa ({username='slave'} at 127.20.210.9:56871)
I20240501 19:56:59.848482 27085 tablet_copy_service.cc:161] P 16cfc5a5082742c5b09d35e5c4f7f10e: Beginning new tablet copy session on tablet 37288cafed764ba18fcc07b6e3c7a722 from peer 045b41d2fb5545fd9975510efab305aa at {username='slave'} at 127.20.210.9:56871: session id = 045b41d2fb5545fd9975510efab305aa-37288cafed764ba18fcc07b6e3c7a722
I20240501 19:56:59.849341 27085 tablet_copy_source_session.cc:205] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:59.850155 32554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37288cafed764ba18fcc07b6e3c7a722. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:59.851963 32554 tablet_copy_client.cc:783] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:59.852097 32554 tablet_copy_client.cc:647] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:59.853077 32557 tablet_copy_client.cc:1002] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:59.924614 32579 ts_tablet_manager.cc:902] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b: Initiating tablet copy from peer ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:59.925103 32579 tablet_copy_client.cc:308] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:43113
I20240501 19:56:59.925530 27217 tablet_copy_service.cc:140] P ef2d33d636574e8fbb9d3b90a15035d6: Received BeginTabletCopySession request for tablet b2c944ca787b475294f9c41f6052a7ea from peer 724bae28280242c6982c4f8e315b0d0b ({username='slave'} at 127.20.210.5:41023)
I20240501 19:56:59.925630 27217 tablet_copy_service.cc:161] P ef2d33d636574e8fbb9d3b90a15035d6: Beginning new tablet copy session on tablet b2c944ca787b475294f9c41f6052a7ea from peer 724bae28280242c6982c4f8e315b0d0b at {username='slave'} at 127.20.210.5:41023: session id = 724bae28280242c6982c4f8e315b0d0b-b2c944ca787b475294f9c41f6052a7ea
I20240501 19:56:59.926316 27217 tablet_copy_source_session.cc:205] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:59.927161 32579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2c944ca787b475294f9c41f6052a7ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:59.929018 32579 tablet_copy_client.cc:783] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:59.929155 32579 tablet_copy_client.cc:647] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:59.930136 32583 tablet_copy_client.cc:1002] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:59.965080 32584 ts_tablet_manager.cc:902] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b: Initiating tablet copy from peer ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:59.965584 32584 tablet_copy_client.cc:308] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:43113
I20240501 19:56:59.966024 27217 tablet_copy_service.cc:140] P ef2d33d636574e8fbb9d3b90a15035d6: Received BeginTabletCopySession request for tablet 896072a285b649dbb31f9f02666ace25 from peer 724bae28280242c6982c4f8e315b0d0b ({username='slave'} at 127.20.210.5:41023)
I20240501 19:56:59.966120 27217 tablet_copy_service.cc:161] P ef2d33d636574e8fbb9d3b90a15035d6: Beginning new tablet copy session on tablet 896072a285b649dbb31f9f02666ace25 from peer 724bae28280242c6982c4f8e315b0d0b at {username='slave'} at 127.20.210.5:41023: session id = 724bae28280242c6982c4f8e315b0d0b-896072a285b649dbb31f9f02666ace25
I20240501 19:56:59.966930 27217 tablet_copy_source_session.cc:205] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:59.967293 32589 ts_tablet_manager.cc:902] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea: Initiating tablet copy from peer ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:56:59.967797 32584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896072a285b649dbb31f9f02666ace25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:59.967851 32589 tablet_copy_client.cc:308] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:43113
I20240501 19:56:59.968309 27217 tablet_copy_service.cc:140] P ef2d33d636574e8fbb9d3b90a15035d6: Received BeginTabletCopySession request for tablet 0f417f4001bc48158187ed262f7737ab from peer 7ccb4fe66ab942d399551b10f5ed32ea ({username='slave'} at 127.20.210.6:37027)
I20240501 19:56:59.968397 27217 tablet_copy_service.cc:161] P ef2d33d636574e8fbb9d3b90a15035d6: Beginning new tablet copy session on tablet 0f417f4001bc48158187ed262f7737ab from peer 7ccb4fe66ab942d399551b10f5ed32ea at {username='slave'} at 127.20.210.6:37027: session id = 7ccb4fe66ab942d399551b10f5ed32ea-0f417f4001bc48158187ed262f7737ab
I20240501 19:56:59.968931 27217 tablet_copy_source_session.cc:205] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:56:59.969617 32589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f417f4001bc48158187ed262f7737ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:56:59.969794 32584 tablet_copy_client.cc:783] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:59.969971 32584 tablet_copy_client.cc:647] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:59.970981 32588 tablet_copy_client.cc:1002] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Injecting latency into file download: 1500
I20240501 19:56:59.971635 32589 tablet_copy_client.cc:783] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 0 data blocks...
I20240501 19:56:59.971819 32589 tablet_copy_client.cc:647] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 1 WAL segments...
I20240501 19:56:59.972817 32593 tablet_copy_client.cc:1002] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Injecting latency into file download: 1500
I20240501 19:57:00.005125 32594 ts_tablet_manager.cc:902] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:57:00.005566 32594 tablet_copy_client.cc:308] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:57:00.006026 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 3e05a10bfc6540c19faa154896b8976a from peer ecdc0d100f5f4b95b9dda10c2f97cf20 ({username='slave'} at 127.20.210.7:35965)
I20240501 19:57:00.006115 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 3e05a10bfc6540c19faa154896b8976a from peer ecdc0d100f5f4b95b9dda10c2f97cf20 at {username='slave'} at 127.20.210.7:35965: session id = ecdc0d100f5f4b95b9dda10c2f97cf20-3e05a10bfc6540c19faa154896b8976a
I20240501 19:57:00.006837 27348 tablet_copy_source_session.cc:205] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:00.007542 32594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e05a10bfc6540c19faa154896b8976a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:00.009059 32594 tablet_copy_client.cc:783] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:00.009196 32594 tablet_copy_client.cc:647] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:00.010017 32598 tablet_copy_client.cc:1002] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Injecting latency into file download: 1500
I20240501 19:57:00.036232 32599 ts_tablet_manager.cc:902] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa: Initiating tablet copy from peer ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113)
I20240501 19:57:00.036700 32599 tablet_copy_client.cc:308] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:43113
I20240501 19:57:00.039853 27217 tablet_copy_service.cc:140] P ef2d33d636574e8fbb9d3b90a15035d6: Received BeginTabletCopySession request for tablet 55d39ed3702046b08b1ac3c76c272c01 from peer 045b41d2fb5545fd9975510efab305aa ({username='slave'} at 127.20.210.9:43593)
I20240501 19:57:00.039953 27217 tablet_copy_service.cc:161] P ef2d33d636574e8fbb9d3b90a15035d6: Beginning new tablet copy session on tablet 55d39ed3702046b08b1ac3c76c272c01 from peer 045b41d2fb5545fd9975510efab305aa at {username='slave'} at 127.20.210.9:43593: session id = 045b41d2fb5545fd9975510efab305aa-55d39ed3702046b08b1ac3c76c272c01
I20240501 19:57:00.040642 27217 tablet_copy_source_session.cc:205] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:00.041349 32599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d39ed3702046b08b1ac3c76c272c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:00.043023 32599 tablet_copy_client.cc:783] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:00.043166 32599 tablet_copy_client.cc:647] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:00.044070 32603 tablet_copy_client.cc:1002] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Injecting latency into file download: 1500
I20240501 19:57:00.094357 32629 ts_tablet_manager.cc:902] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa: Initiating tablet copy from peer e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:57:00.094852 32629 tablet_copy_client.cc:308] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:37115
I20240501 19:57:00.096727 32639 ts_tablet_manager.cc:902] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b: Initiating tablet copy from peer 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:57:00.097204 32639 tablet_copy_client.cc:308] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:33375
I20240501 19:57:00.097810 27085 tablet_copy_service.cc:140] P 16cfc5a5082742c5b09d35e5c4f7f10e: Received BeginTabletCopySession request for tablet 199f9529870242f599358995c19250b2 from peer 724bae28280242c6982c4f8e315b0d0b ({username='slave'} at 127.20.210.5:49029)
I20240501 19:57:00.097903 27085 tablet_copy_service.cc:161] P 16cfc5a5082742c5b09d35e5c4f7f10e: Beginning new tablet copy session on tablet 199f9529870242f599358995c19250b2 from peer 724bae28280242c6982c4f8e315b0d0b at {username='slave'} at 127.20.210.5:49029: session id = 724bae28280242c6982c4f8e315b0d0b-199f9529870242f599358995c19250b2
I20240501 19:57:00.098731 27085 tablet_copy_source_session.cc:205] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:00.099180 27479 tablet_copy_service.cc:140] P e5057264da53436ea17bad01c66a7fbf: Received BeginTabletCopySession request for tablet 9102f7c8350843c2bd7009ea190860d0 from peer 045b41d2fb5545fd9975510efab305aa ({username='slave'} at 127.20.210.9:36467)
I20240501 19:57:00.099293 27479 tablet_copy_service.cc:161] P e5057264da53436ea17bad01c66a7fbf: Beginning new tablet copy session on tablet 9102f7c8350843c2bd7009ea190860d0 from peer 045b41d2fb5545fd9975510efab305aa at {username='slave'} at 127.20.210.9:36467: session id = 045b41d2fb5545fd9975510efab305aa-9102f7c8350843c2bd7009ea190860d0
I20240501 19:57:00.099483 32639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199f9529870242f599358995c19250b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:00.100118 27479 tablet_copy_source_session.cc:205] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:00.100823 32629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9102f7c8350843c2bd7009ea190860d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:00.101416 32639 tablet_copy_client.cc:783] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:00.101550 32639 tablet_copy_client.cc:647] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:00.102706 32629 tablet_copy_client.cc:783] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:00.102823 32640 tablet_copy_client.cc:1002] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Injecting latency into file download: 1500
I20240501 19:57:00.102850 32629 tablet_copy_client.cc:647] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:00.103708 32637 tablet_copy_client.cc:1002] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Injecting latency into file download: 1500
I20240501 19:57:01.249238 32470 tablet_copy_client.cc:523] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.250574 32470 tablet_bootstrap.cc:492] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b: Bootstrap starting.
I20240501 19:57:01.266520 32470 tablet_bootstrap.cc:492] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.266788 32470 tablet_bootstrap.cc:492] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b: Bootstrap complete.
I20240501 19:57:01.266968 32470 ts_tablet_manager.cc:1367] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b: Time spent bootstrapping tablet: real 0.016s user 0.016s sys 0.000s
I20240501 19:57:01.267385 32470 raft_consensus.cc:348] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:57:01.267520 32470 raft_consensus.cc:729] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 724bae28280242c6982c4f8e315b0d0b, State: Initialized, Role: LEARNER
I20240501 19:57:01.267623 32470 consensus_queue.cc:260] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:57:01.267992 32470 ts_tablet_manager.cc:1398] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 19:57:01.268450 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session 724bae28280242c6982c4f8e315b0d0b-f149fd8790a0412e81ac49d0ce359ede received from {username='slave'} at 127.20.210.5:40583
I20240501 19:57:01.268533 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session 724bae28280242c6982c4f8e315b0d0b-f149fd8790a0412e81ac49d0ce359ede on tablet f149fd8790a0412e81ac49d0ce359ede with peer 724bae28280242c6982c4f8e315b0d0b
I20240501 19:57:01.269503 28746 raft_consensus.cc:1203] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:01.273100 32475 tablet_copy_client.cc:523] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.274220 32475 tablet_bootstrap.cc:492] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap starting.
I20240501 19:57:01.290910 32475 tablet_bootstrap.cc:492] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.291162 32475 tablet_bootstrap.cc:492] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap complete.
I20240501 19:57:01.291297 32475 ts_tablet_manager.cc:1367] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.004s
I20240501 19:57:01.291599 32475 raft_consensus.cc:348] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:57:01.291690 32475 raft_consensus.cc:729] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ecdc0d100f5f4b95b9dda10c2f97cf20, State: Initialized, Role: LEARNER
I20240501 19:57:01.291791 32475 consensus_queue.cc:260] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:57:01.292032 32475 ts_tablet_manager.cc:1398] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:01.292492 27479 tablet_copy_service.cc:342] P e5057264da53436ea17bad01c66a7fbf: Request end of tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-3ca822a1462544c3be45371d6e088447 received from {username='slave'} at 127.20.210.7:47243
I20240501 19:57:01.292577 27479 tablet_copy_service.cc:433] P e5057264da53436ea17bad01c66a7fbf: ending tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-3ca822a1462544c3be45371d6e088447 on tablet 3ca822a1462544c3be45371d6e088447 with peer ecdc0d100f5f4b95b9dda10c2f97cf20
I20240501 19:57:01.294847 32481 tablet_copy_client.cc:523] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.295855 32481 tablet_bootstrap.cc:492] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap starting.
I20240501 19:57:01.311249 32502 tablet_copy_client.cc:523] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.312398 32502 tablet_bootstrap.cc:492] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap starting.
I20240501 19:57:01.314453 32481 tablet_bootstrap.cc:492] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.314709 32481 tablet_bootstrap.cc:492] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap complete.
I20240501 19:57:01.314862 32481 ts_tablet_manager.cc:1367] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b: Time spent bootstrapping tablet: real 0.019s user 0.015s sys 0.005s
I20240501 19:57:01.315212 32481 raft_consensus.cc:348] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:57:01.315325 32481 raft_consensus.cc:729] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 724bae28280242c6982c4f8e315b0d0b, State: Initialized, Role: LEARNER
I20240501 19:57:01.315407 32481 consensus_queue.cc:260] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:57:01.315631 32481 ts_tablet_manager.cc:1398] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:01.316102 27085 tablet_copy_service.cc:342] P 16cfc5a5082742c5b09d35e5c4f7f10e: Request end of tablet copy session 724bae28280242c6982c4f8e315b0d0b-6c498eacc5e947bfb01dfb90c7039df0 received from {username='slave'} at 127.20.210.5:49029
I20240501 19:57:01.316186 27085 tablet_copy_service.cc:433] P 16cfc5a5082742c5b09d35e5c4f7f10e: ending tablet copy session 724bae28280242c6982c4f8e315b0d0b-6c498eacc5e947bfb01dfb90c7039df0 on tablet 6c498eacc5e947bfb01dfb90c7039df0 with peer 724bae28280242c6982c4f8e315b0d0b
I20240501 19:57:01.322779 28746 raft_consensus.cc:1203] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:01.324635 32507 tablet_copy_client.cc:523] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.326333 32507 tablet_bootstrap.cc:492] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa: Bootstrap starting.
I20240501 19:57:01.330826 32502 tablet_bootstrap.cc:492] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.331063 32502 tablet_bootstrap.cc:492] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1: Bootstrap complete.
I20240501 19:57:01.331218 32502 ts_tablet_manager.cc:1367] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.004s
I20240501 19:57:01.331535 32523 tablet_copy_client.cc:523] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.332190 32507 log.cc:824] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:01.332580 32523 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa: Bootstrap starting.
I20240501 19:57:01.337785 32502 raft_consensus.cc:348] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:57:01.337939 32502 raft_consensus.cc:729] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 474eb4491dcb47f4a2a2f6e149bc39c1, State: Initialized, Role: LEARNER
I20240501 19:57:01.338038 32502 consensus_queue.cc:260] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: true } }
I20240501 19:57:01.338339 32502 ts_tablet_manager.cc:1398] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20240501 19:57:01.338514 29008 raft_consensus.cc:1203] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:01.338762 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-0aaa59df03844aadafb8d57797685dc2 received from {username='slave'} at 127.20.210.8:56643
I20240501 19:57:01.338829 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session 474eb4491dcb47f4a2a2f6e149bc39c1-0aaa59df03844aadafb8d57797685dc2 on tablet 0aaa59df03844aadafb8d57797685dc2 with peer 474eb4491dcb47f4a2a2f6e149bc39c1
I20240501 19:57:01.353324 32554 tablet_copy_client.cc:523] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.354589 32554 tablet_bootstrap.cc:492] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa: Bootstrap starting.
I20240501 19:57:01.356076 32507 tablet_bootstrap.cc:492] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.356099 32523 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.356323 32523 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa: Bootstrap complete.
I20240501 19:57:01.356359 32507 tablet_bootstrap.cc:492] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa: Bootstrap complete.
I20240501 19:57:01.356527 32507 ts_tablet_manager.cc:1367] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa: Time spent bootstrapping tablet: real 0.030s user 0.026s sys 0.001s
I20240501 19:57:01.356530 32523 ts_tablet_manager.cc:1367] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa: Time spent bootstrapping tablet: real 0.024s user 0.019s sys 0.002s
I20240501 19:57:01.358512 32507 raft_consensus.cc:348] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:01.358515 32523 raft_consensus.cc:348] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:01.358723 32507 raft_consensus.cc:729] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 045b41d2fb5545fd9975510efab305aa, State: Initialized, Role: LEARNER
I20240501 19:57:01.358723 32523 raft_consensus.cc:729] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 045b41d2fb5545fd9975510efab305aa, State: Initialized, Role: LEARNER
I20240501 19:57:01.359093 32507 consensus_queue.cc:260] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:01.359093 32523 consensus_queue.cc:260] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:01.359841 32523 ts_tablet_manager.cc:1398] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:01.361930 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session 045b41d2fb5545fd9975510efab305aa-5a9674523a2148b1a6c1a5d22a57aac5 received from {username='slave'} at 127.20.210.9:37755
I20240501 19:57:01.362010 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session 045b41d2fb5545fd9975510efab305aa-5a9674523a2148b1a6c1a5d22a57aac5 on tablet 5a9674523a2148b1a6c1a5d22a57aac5 with peer 045b41d2fb5545fd9975510efab305aa
I20240501 19:57:01.362532 32507 ts_tablet_manager.cc:1398] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240501 19:57:01.362939 27085 tablet_copy_service.cc:342] P 16cfc5a5082742c5b09d35e5c4f7f10e: Request end of tablet copy session 045b41d2fb5545fd9975510efab305aa-c9f961f0554c45a1a3f31d5a8b31a46d received from {username='slave'} at 127.20.210.9:56871
I20240501 19:57:01.363009 27085 tablet_copy_service.cc:433] P 16cfc5a5082742c5b09d35e5c4f7f10e: ending tablet copy session 045b41d2fb5545fd9975510efab305aa-c9f961f0554c45a1a3f31d5a8b31a46d on tablet c9f961f0554c45a1a3f31d5a8b31a46d with peer 045b41d2fb5545fd9975510efab305aa
I20240501 19:57:01.370357 29140 raft_consensus.cc:1203] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:01.374500 32554 tablet_bootstrap.cc:492] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.374688 32554 tablet_bootstrap.cc:492] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa: Bootstrap complete.
I20240501 19:57:01.374809 32554 ts_tablet_manager.cc:1367] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20240501 19:57:01.375062 32554 raft_consensus.cc:348] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:01.375137 32554 raft_consensus.cc:729] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 045b41d2fb5545fd9975510efab305aa, State: Initialized, Role: LEARNER
I20240501 19:57:01.375216 32554 consensus_queue.cc:260] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:01.375406 32554 ts_tablet_manager.cc:1398] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:01.375756 27085 tablet_copy_service.cc:342] P 16cfc5a5082742c5b09d35e5c4f7f10e: Request end of tablet copy session 045b41d2fb5545fd9975510efab305aa-37288cafed764ba18fcc07b6e3c7a722 received from {username='slave'} at 127.20.210.9:56871
I20240501 19:57:01.375829 27085 tablet_copy_service.cc:433] P 16cfc5a5082742c5b09d35e5c4f7f10e: ending tablet copy session 045b41d2fb5545fd9975510efab305aa-37288cafed764ba18fcc07b6e3c7a722 on tablet 37288cafed764ba18fcc07b6e3c7a722 with peer 045b41d2fb5545fd9975510efab305aa
I20240501 19:57:01.388569 30036 raft_consensus.cc:1203] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:01.430410 32579 tablet_copy_client.cc:523] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.431573 32579 tablet_bootstrap.cc:492] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b: Bootstrap starting.
I20240501 19:57:01.453613 32579 tablet_bootstrap.cc:492] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.453898 32579 tablet_bootstrap.cc:492] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b: Bootstrap complete.
I20240501 19:57:01.454064 32579 ts_tablet_manager.cc:1367] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b: Time spent bootstrapping tablet: real 0.023s user 0.016s sys 0.005s
I20240501 19:57:01.454442 32579 raft_consensus.cc:348] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:57:01.454553 32579 raft_consensus.cc:729] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 724bae28280242c6982c4f8e315b0d0b, State: Initialized, Role: LEARNER
I20240501 19:57:01.454640 32579 consensus_queue.cc:260] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:57:01.454926 32579 ts_tablet_manager.cc:1398] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 19:57:01.455356 27217 tablet_copy_service.cc:342] P ef2d33d636574e8fbb9d3b90a15035d6: Request end of tablet copy session 724bae28280242c6982c4f8e315b0d0b-b2c944ca787b475294f9c41f6052a7ea received from {username='slave'} at 127.20.210.5:41023
I20240501 19:57:01.455431 27217 tablet_copy_service.cc:433] P ef2d33d636574e8fbb9d3b90a15035d6: ending tablet copy session 724bae28280242c6982c4f8e315b0d0b-b2c944ca787b475294f9c41f6052a7ea on tablet b2c944ca787b475294f9c41f6052a7ea with peer 724bae28280242c6982c4f8e315b0d0b
I20240501 19:57:01.471225 32584 tablet_copy_client.cc:523] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.472405 32584 tablet_bootstrap.cc:492] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap starting.
I20240501 19:57:01.473063 32589 tablet_copy_client.cc:523] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.474468 32589 tablet_bootstrap.cc:492] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap starting.
I20240501 19:57:01.497862 32584 tablet_bootstrap.cc:492] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.498083 32584 tablet_bootstrap.cc:492] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap complete.
I20240501 19:57:01.498252 32584 ts_tablet_manager.cc:1367] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b: Time spent bootstrapping tablet: real 0.026s user 0.025s sys 0.000s
I20240501 19:57:01.498665 32584 raft_consensus.cc:348] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:57:01.498783 32584 raft_consensus.cc:729] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 724bae28280242c6982c4f8e315b0d0b, State: Initialized, Role: LEARNER
I20240501 19:57:01.498886 32584 consensus_queue.cc:260] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:57:01.500692 32584 ts_tablet_manager.cc:1398] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:01.500873 32589 tablet_bootstrap.cc:492] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.501389 32589 tablet_bootstrap.cc:492] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap complete.
I20240501 19:57:01.501695 32589 ts_tablet_manager.cc:1367] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent bootstrapping tablet: real 0.027s user 0.023s sys 0.000s
I20240501 19:57:01.502066 32589 raft_consensus.cc:348] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:57:01.502177 32589 raft_consensus.cc:729] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7ccb4fe66ab942d399551b10f5ed32ea, State: Initialized, Role: LEARNER
I20240501 19:57:01.502149 27217 tablet_copy_service.cc:342] P ef2d33d636574e8fbb9d3b90a15035d6: Request end of tablet copy session 724bae28280242c6982c4f8e315b0d0b-896072a285b649dbb31f9f02666ace25 received from {username='slave'} at 127.20.210.5:41023
I20240501 19:57:01.502228 27217 tablet_copy_service.cc:433] P ef2d33d636574e8fbb9d3b90a15035d6: ending tablet copy session 724bae28280242c6982c4f8e315b0d0b-896072a285b649dbb31f9f02666ace25 on tablet 896072a285b649dbb31f9f02666ace25 with peer 724bae28280242c6982c4f8e315b0d0b
I20240501 19:57:01.502310 32589 consensus_queue.cc:260] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:57:01.502637 32589 ts_tablet_manager.cc:1398] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:01.503432 27217 tablet_copy_service.cc:342] P ef2d33d636574e8fbb9d3b90a15035d6: Request end of tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-0f417f4001bc48158187ed262f7737ab received from {username='slave'} at 127.20.210.6:37027
I20240501 19:57:01.503507 27217 tablet_copy_service.cc:433] P ef2d33d636574e8fbb9d3b90a15035d6: ending tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-0f417f4001bc48158187ed262f7737ab on tablet 0f417f4001bc48158187ed262f7737ab with peer 7ccb4fe66ab942d399551b10f5ed32ea
I20240501 19:57:01.510357 32594 tablet_copy_client.cc:523] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.511629 32594 tablet_bootstrap.cc:492] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap starting.
I20240501 19:57:01.529966 32594 tablet_bootstrap.cc:492] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.530256 32594 tablet_bootstrap.cc:492] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20: Bootstrap complete.
I20240501 19:57:01.530433 32594 ts_tablet_manager.cc:1367] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent bootstrapping tablet: real 0.019s user 0.020s sys 0.000s
I20240501 19:57:01.530815 32594 raft_consensus.cc:348] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:57:01.530913 32594 raft_consensus.cc:729] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ecdc0d100f5f4b95b9dda10c2f97cf20, State: Initialized, Role: LEARNER
I20240501 19:57:01.531005 32594 consensus_queue.cc:260] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: NON_VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: true } }
I20240501 19:57:01.531240 32594 ts_tablet_manager.cc:1398] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:01.531693 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-3e05a10bfc6540c19faa154896b8976a received from {username='slave'} at 127.20.210.7:35965
I20240501 19:57:01.531785 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session ecdc0d100f5f4b95b9dda10c2f97cf20-3e05a10bfc6540c19faa154896b8976a on tablet 3e05a10bfc6540c19faa154896b8976a with peer ecdc0d100f5f4b95b9dda10c2f97cf20
I20240501 19:57:01.544338 32599 tablet_copy_client.cc:523] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.545516 32599 tablet_bootstrap.cc:492] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa: Bootstrap starting.
I20240501 19:57:01.562393 32599 tablet_bootstrap.cc:492] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.562636 32599 tablet_bootstrap.cc:492] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa: Bootstrap complete.
I20240501 19:57:01.562819 32599 ts_tablet_manager.cc:1367] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.003s
I20240501 19:57:01.563196 32599 raft_consensus.cc:348] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:01.563318 32599 raft_consensus.cc:729] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 045b41d2fb5545fd9975510efab305aa, State: Initialized, Role: LEARNER
I20240501 19:57:01.563386 32599 consensus_queue.cc:260] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:01.563627 32599 ts_tablet_manager.cc:1398] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:01.564132 27217 tablet_copy_service.cc:342] P ef2d33d636574e8fbb9d3b90a15035d6: Request end of tablet copy session 045b41d2fb5545fd9975510efab305aa-55d39ed3702046b08b1ac3c76c272c01 received from {username='slave'} at 127.20.210.9:43593
I20240501 19:57:01.564222 27217 tablet_copy_service.cc:433] P ef2d33d636574e8fbb9d3b90a15035d6: ending tablet copy session 045b41d2fb5545fd9975510efab305aa-55d39ed3702046b08b1ac3c76c272c01 on tablet 55d39ed3702046b08b1ac3c76c272c01 with peer 045b41d2fb5545fd9975510efab305aa
I20240501 19:57:01.599223 28746 raft_consensus.cc:1203] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:01.603052 32639 tablet_copy_client.cc:523] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.604000 32629 tablet_copy_client.cc:523] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:01.604279 32639 tablet_bootstrap.cc:492] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap starting.
I20240501 19:57:01.605332 32629 tablet_bootstrap.cc:492] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa: Bootstrap starting.
I20240501 19:57:01.606609 30036 raft_consensus.cc:1203] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:01.622082 32639 tablet_bootstrap.cc:492] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.622319 32639 tablet_bootstrap.cc:492] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b: Bootstrap complete.
I20240501 19:57:01.622335 32629 tablet_bootstrap.cc:492] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:01.622463 32639 ts_tablet_manager.cc:1367] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.008s
I20240501 19:57:01.622517 32629 tablet_bootstrap.cc:492] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa: Bootstrap complete.
I20240501 19:57:01.622648 32629 ts_tablet_manager.cc:1367] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa: Time spent bootstrapping tablet: real 0.017s user 0.017s sys 0.001s
I20240501 19:57:01.622860 32639 raft_consensus.cc:348] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:57:01.622953 32639 raft_consensus.cc:729] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 724bae28280242c6982c4f8e315b0d0b, State: Initialized, Role: LEARNER
I20240501 19:57:01.623005 32629 raft_consensus.cc:348] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:01.623050 32639 consensus_queue.cc:260] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: true } }
I20240501 19:57:01.623127 32629 raft_consensus.cc:729] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 045b41d2fb5545fd9975510efab305aa, State: Initialized, Role: LEARNER
I20240501 19:57:01.623241 32629 consensus_queue.cc:260] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:01.623483 32629 ts_tablet_manager.cc:1398] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:01.623998 27479 tablet_copy_service.cc:342] P e5057264da53436ea17bad01c66a7fbf: Request end of tablet copy session 045b41d2fb5545fd9975510efab305aa-9102f7c8350843c2bd7009ea190860d0 received from {username='slave'} at 127.20.210.9:36467
I20240501 19:57:01.624081 27479 tablet_copy_service.cc:433] P e5057264da53436ea17bad01c66a7fbf: ending tablet copy session 045b41d2fb5545fd9975510efab305aa-9102f7c8350843c2bd7009ea190860d0 on tablet 9102f7c8350843c2bd7009ea190860d0 with peer 045b41d2fb5545fd9975510efab305aa
I20240501 19:57:01.625062 32639 ts_tablet_manager.cc:1398] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240501 19:57:01.625584 27085 tablet_copy_service.cc:342] P 16cfc5a5082742c5b09d35e5c4f7f10e: Request end of tablet copy session 724bae28280242c6982c4f8e315b0d0b-199f9529870242f599358995c19250b2 received from {username='slave'} at 127.20.210.5:49029
I20240501 19:57:01.625646 27085 tablet_copy_service.cc:433] P 16cfc5a5082742c5b09d35e5c4f7f10e: ending tablet copy session 724bae28280242c6982c4f8e315b0d0b-199f9529870242f599358995c19250b2 on tablet 199f9529870242f599358995c19250b2 with peer 724bae28280242c6982c4f8e315b0d0b
I20240501 19:57:01.709542 28746 raft_consensus.cc:1203] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:01.722329 601 raft_consensus.cc:1050] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf: attempting to promote NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 to VOTER
I20240501 19:57:01.722764 601 consensus_queue.cc:237] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:57:01.724923 29139 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.724931 27066 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.725293 29008 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.725320 32687 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.725659 32687 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.725869 601 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.728109 601 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:01.728685 27066 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:01.728876 29139 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:01.731386 26915 catalog_manager.cc:5484] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: config changed from index 7 to 8, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:01.731423 29008 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:01.735349 27459 consensus_queue.cc:237] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:57:01.736371 29008 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:01.736413 29139 raft_consensus.cc:1261] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:01.736714 604 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:01.736896 32687 consensus_queue.cc:1035] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:01.738610 600 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:01.739084 29139 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:01.739091 29008 raft_consensus.cc:2932] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:01.740267 26911 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3ca822a1462544c3be45371d6e088447 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:01.741099 26918 catalog_manager.cc:5484] T 3ca822a1462544c3be45371d6e088447 P ecdc0d100f5f4b95b9dda10c2f97cf20 reported cstate change: config changed from index 8 to 9, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:01.743360 27045 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ca822a1462544c3be45371d6e088447 with delete_type TABLET_DATA_TOMBSTONED (TS 16cfc5a5082742c5b09d35e5c4f7f10e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:49644
I20240501 19:57:01.743993 624 tablet_replica.cc:330] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e: stopping tablet replica
I20240501 19:57:01.744189 624 raft_consensus.cc:2229] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:01.744448 624 raft_consensus.cc:2258] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:01.745759 624 ts_tablet_manager.cc:1875] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:01.747777 624 ts_tablet_manager.cc:1888] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:01.747839 624 log.cc:1196] T 3ca822a1462544c3be45371d6e088447 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/3ca822a1462544c3be45371d6e088447
I20240501 19:57:01.748392 26909 catalog_manager.cc:4830] TS 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375): tablet 3ca822a1462544c3be45371d6e088447 (table rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]) successfully deleted
I20240501 19:57:01.782889 663 raft_consensus.cc:1050] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: attempting to promote NON_VOTER 724bae28280242c6982c4f8e315b0d0b to VOTER
I20240501 19:57:01.783354 663 consensus_queue.cc:237] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:57:01.784806 29008 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.784814 28746 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.785051 27197 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.785537 31562 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.785691 32684 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.786059 32684 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.788697 673 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:01.788902 29008 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:01.788990 27196 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:01.791574 26924 catalog_manager.cc:5484] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 reported cstate change: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:01.792253 28746 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:01.793543 30036 raft_consensus.cc:1203] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:01.802356 30036 raft_consensus.cc:1203] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:01.807154 32689 raft_consensus.cc:1050] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER 045b41d2fb5545fd9975510efab305aa to VOTER
I20240501 19:57:01.807554 32689 consensus_queue.cc:237] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:01.809340 27196 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.809877 691 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.810531 28746 raft_consensus.cc:1203] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:01.811129 29139 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.812104 30036 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.812094 691 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.812748 691 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.815292 692 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:01.816906 27196 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:01.818935 30036 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:01.819870 26916 catalog_manager.cc:5484] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:01.821286 29139 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:01.830034 691 raft_consensus.cc:1050] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER 724bae28280242c6982c4f8e315b0d0b to VOTER
I20240501 19:57:01.830425 691 consensus_queue.cc:237] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:57:01.831660 29008 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.831686 27066 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.831992 692 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.832154 691 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.832365 28746 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.832646 691 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.834396 689 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:01.835093 28746 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:01.835109 29008 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:01.836359 27066 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:01.837678 26915 catalog_manager.cc:5484] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b reported cstate change: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:01.841358 27328 consensus_queue.cc:237] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:57:01.842464 28746 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:01.842491 29008 raft_consensus.cc:1261] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:01.842767 692 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:01.843055 691 consensus_queue.cc:1035] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:01.844457 692 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:01.844952 28746 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:01.845816 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f149fd8790a0412e81ac49d0ce359ede with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:01.846612 29008 raft_consensus.cc:2932] T f149fd8790a0412e81ac49d0ce359ede P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:01.846803 26921 catalog_manager.cc:5484] T f149fd8790a0412e81ac49d0ce359ede P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 8 to 9, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:01.849510 27045 tablet_service.cc:1514] Processing DeleteTablet for tablet f149fd8790a0412e81ac49d0ce359ede with delete_type TABLET_DATA_TOMBSTONED (TS 16cfc5a5082742c5b09d35e5c4f7f10e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:49644
I20240501 19:57:01.849686 624 tablet_replica.cc:330] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e: stopping tablet replica
I20240501 19:57:01.849777 624 raft_consensus.cc:2229] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:01.849937 624 raft_consensus.cc:2258] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:01.850562 624 ts_tablet_manager.cc:1875] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:01.852017 624 ts_tablet_manager.cc:1888] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:01.852078 624 log.cc:1196] T f149fd8790a0412e81ac49d0ce359ede P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/f149fd8790a0412e81ac49d0ce359ede
I20240501 19:57:01.852669 26909 catalog_manager.cc:4830] TS 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375): tablet f149fd8790a0412e81ac49d0ce359ede (table rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]) successfully deleted
I20240501 19:57:01.856962 27328 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5a9674523a2148b1a6c1a5d22a57aac5"
dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:60064
I20240501 19:57:01.857061 27328 raft_consensus.cc:595] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:57:01.865307 27066 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6c498eacc5e947bfb01dfb90c7039df0"
dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:52596
I20240501 19:57:01.865437 27066 raft_consensus.cc:595] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Received request to transfer leadership
I20240501 19:57:01.870556 29008 raft_consensus.cc:1203] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:01.881301 32689 raft_consensus.cc:1050] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER 474eb4491dcb47f4a2a2f6e149bc39c1 to VOTER
I20240501 19:57:01.881633 32689 consensus_queue.cc:237] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:57:01.882921 29139 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.882972 27459 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.883016 28876 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:01.883373 32689 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.883529 692 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.885279 32689 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:01.886847 692 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:57:01.887380 27459 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:57:01.887380 29139 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:57:01.889222 28876 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:57:01.890056 26915 catalog_manager.cc:5484] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 7 to 8, 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:57:01.934767 27328 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5a9674523a2148b1a6c1a5d22a57aac5"
dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:60116
I20240501 19:57:01.934891 27328 raft_consensus.cc:595] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:57:01.943606 27066 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6c498eacc5e947bfb01dfb90c7039df0"
dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:52648
I20240501 19:57:01.943708 27066 raft_consensus.cc:595] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Received request to transfer leadership
I20240501 19:57:01.961572 27328 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0aaa59df03844aadafb8d57797685dc2"
dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:60128
I20240501 19:57:01.961680 27328 raft_consensus.cc:595] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:57:02.013337 28876 raft_consensus.cc:1203] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:02.046788 30036 raft_consensus.cc:1203] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:02.112138 851 raft_consensus.cc:1050] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: attempting to promote NON_VOTER 045b41d2fb5545fd9975510efab305aa to VOTER
I20240501 19:57:02.112496 851 consensus_queue.cc:237] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:02.113871 27066 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.113910 30036 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.113998 29008 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.114257 850 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.114425 851 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.114637 853 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.116497 853 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.116950 29008 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.117280 30036 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.119391 26921 catalog_manager.cc:5484] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 reported cstate change: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.119472 27066 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.184253 32684 raft_consensus.cc:979] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: : Instructing follower ecdc0d100f5f4b95b9dda10c2f97cf20 to start an election
I20240501 19:57:02.184376 32684 raft_consensus.cc:1067] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Signalling peer ecdc0d100f5f4b95b9dda10c2f97cf20 to start an election
I20240501 19:57:02.184949 29008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6c498eacc5e947bfb01dfb90c7039df0"
dest_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20"
from {username='slave'} at 127.20.210.1:42555
I20240501 19:57:02.185079 29008 raft_consensus.cc:482] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 19:57:02.185145 29008 raft_consensus.cc:3037] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.186127 29008 raft_consensus.cc:504] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:57:02.186643 29008 leader_election.cc:288] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [CANDIDATE]: Term 2 election: Requested vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 724bae28280242c6982c4f8e315b0d0b (127.20.210.5:42343)
I20240501 19:57:02.186933 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c498eacc5e947bfb01dfb90c7039df0" candidate_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:57:02.187034 27066 raft_consensus.cc:3032] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Stepping down as leader of term 1
I20240501 19:57:02.187069 27066 raft_consensus.cc:729] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Running, Role: LEADER
I20240501 19:57:02.187247 27066 consensus_queue.cc:260] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:57:02.187469 27066 raft_consensus.cc:3037] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.187969 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c498eacc5e947bfb01dfb90c7039df0" candidate_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:57:02.188079 27196 raft_consensus.cc:3037] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.188408 27066 raft_consensus.cc:2454] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecdc0d100f5f4b95b9dda10c2f97cf20 in term 2.
I20240501 19:57:02.189065 27196 raft_consensus.cc:2454] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecdc0d100f5f4b95b9dda10c2f97cf20 in term 2.
I20240501 19:57:02.189352 28941 leader_election.cc:302] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, ecdc0d100f5f4b95b9dda10c2f97cf20, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:57:02.189510 30462 raft_consensus.cc:2781] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 19:57:02.189651 30462 raft_consensus.cc:686] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 LEADER]: Becoming Leader. State: Replica: ecdc0d100f5f4b95b9dda10c2f97cf20, State: Running, Role: LEADER
I20240501 19:57:02.189800 30462 consensus_queue.cc:237] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:57:02.191951 26915 catalog_manager.cc:5484] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 reported cstate change: term changed from 1 to 2, leader changed from 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) to ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7). New cstate: current_term: 2 leader_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:02.192482 28746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c498eacc5e947bfb01dfb90c7039df0" candidate_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "724bae28280242c6982c4f8e315b0d0b"
I20240501 19:57:02.192624 28746 raft_consensus.cc:3037] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.193506 28746 raft_consensus.cc:2454] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecdc0d100f5f4b95b9dda10c2f97cf20 in term 2.
I20240501 19:57:02.202564 851 raft_consensus.cc:1050] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6: attempting to promote NON_VOTER 724bae28280242c6982c4f8e315b0d0b to VOTER
I20240501 19:57:02.202971 851 consensus_queue.cc:237] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:57:02.204313 27459 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.204468 28746 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.204612 29139 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.204731 32691 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.205744 850 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.206046 850 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.208016 27196 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "55d39ed3702046b08b1ac3c76c272c01"
dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:50334
I20240501 19:57:02.208118 27196 raft_consensus.cc:595] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:57:02.208444 32691 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.209054 28746 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.209461 29139 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.211870 26916 catalog_manager.cc:5484] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b reported cstate change: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.211925 27459 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.215638 27196 consensus_queue.cc:237] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:57:02.216713 29139 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.216760 28746 raft_consensus.cc:1261] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.217219 851 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.217836 853 consensus_queue.cc:1035] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.219345 850 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.220062 29139 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.220757 26908 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 896072a285b649dbb31f9f02666ace25 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:02.222041 28746 raft_consensus.cc:2932] T 896072a285b649dbb31f9f02666ace25 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.222491 26921 catalog_manager.cc:5484] T 896072a285b649dbb31f9f02666ace25 P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:02.224732 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet 896072a285b649dbb31f9f02666ace25 with delete_type TABLET_DATA_TOMBSTONED (TS e5057264da53436ea17bad01c66a7fbf not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41962
I20240501 19:57:02.225231 904 tablet_replica.cc:330] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf: stopping tablet replica
I20240501 19:57:02.225430 904 raft_consensus.cc:2229] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:02.225492 27328 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5a9674523a2148b1a6c1a5d22a57aac5"
dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:60154
I20240501 19:57:02.225600 27328 raft_consensus.cc:595] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:57:02.225685 904 raft_consensus.cc:2258] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:02.227056 904 ts_tablet_manager.cc:1875] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:02.228873 904 ts_tablet_manager.cc:1888] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:02.228937 904 log.cc:1196] T 896072a285b649dbb31f9f02666ace25 P e5057264da53436ea17bad01c66a7fbf: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/896072a285b649dbb31f9f02666ace25
I20240501 19:57:02.229391 26911 catalog_manager.cc:4830] TS e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115): tablet 896072a285b649dbb31f9f02666ace25 (table rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]) successfully deleted
I20240501 19:57:02.243261 32689 raft_consensus.cc:979] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: : Instructing follower 474eb4491dcb47f4a2a2f6e149bc39c1 to start an election
I20240501 19:57:02.243357 32689 raft_consensus.cc:1067] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Signalling peer 474eb4491dcb47f4a2a2f6e149bc39c1 to start an election
I20240501 19:57:02.243860 29139 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5a9674523a2148b1a6c1a5d22a57aac5"
dest_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1"
from {username='slave'} at 127.20.210.3:56397
I20240501 19:57:02.244057 29139 raft_consensus.cc:482] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 19:57:02.244180 29139 raft_consensus.cc:3037] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.245082 29139 raft_consensus.cc:504] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:02.246311 29139 leader_election.cc:288] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [CANDIDATE]: Term 2 election: Requested vote from peers ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833)
I20240501 19:57:02.246783 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9674523a2148b1a6c1a5d22a57aac5" candidate_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:57:02.246918 27196 raft_consensus.cc:3037] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.247017 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9674523a2148b1a6c1a5d22a57aac5" candidate_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:57:02.247133 27328 raft_consensus.cc:3032] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 19:57:02.247174 27328 raft_consensus.cc:729] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:57:02.247306 27328 consensus_queue.cc:260] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:02.247500 27328 raft_consensus.cc:3037] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.247936 27196 raft_consensus.cc:2454] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 474eb4491dcb47f4a2a2f6e149bc39c1 in term 2.
I20240501 19:57:02.248281 27328 raft_consensus.cc:2454] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 474eb4491dcb47f4a2a2f6e149bc39c1 in term 2.
I20240501 19:57:02.248558 29075 leader_election.cc:302] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, 474eb4491dcb47f4a2a2f6e149bc39c1, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:57:02.248782 512 raft_consensus.cc:2781] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 19:57:02.248927 512 raft_consensus.cc:686] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 2 LEADER]: Becoming Leader. State: Replica: 474eb4491dcb47f4a2a2f6e149bc39c1, State: Running, Role: LEADER
I20240501 19:57:02.249173 512 consensus_queue.cc:237] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:02.250977 30036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a9674523a2148b1a6c1a5d22a57aac5" candidate_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "045b41d2fb5545fd9975510efab305aa"
I20240501 19:57:02.251137 30036 raft_consensus.cc:3037] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.251760 26918 catalog_manager.cc:5484] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: term changed from 1 to 2, leader changed from 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) to 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8). New cstate: current_term: 2 leader_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:02.252068 30036 raft_consensus.cc:2454] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 474eb4491dcb47f4a2a2f6e149bc39c1 in term 2.
I20240501 19:57:02.252892 27328 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0aaa59df03844aadafb8d57797685dc2"
dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:60172
I20240501 19:57:02.252995 27328 raft_consensus.cc:595] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:57:02.284360 32691 raft_consensus.cc:1050] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6: attempting to promote NON_VOTER 724bae28280242c6982c4f8e315b0d0b to VOTER
I20240501 19:57:02.284811 32691 consensus_queue.cc:237] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:57:02.286074 28876 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.286094 27459 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.286123 28746 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.286522 850 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.287119 850 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.287289 851 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.289080 853 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.289567 28876 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.289634 28746 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.291553 26918 catalog_manager.cc:5484] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:02.291707 27459 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.295017 27196 consensus_queue.cc:237] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:57:02.295918 28746 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.295923 28876 raft_consensus.cc:1261] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.296203 853 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.296345 851 consensus_queue.cc:1035] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.297816 850 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.298285 28746 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.298492 28876 raft_consensus.cc:2932] T b2c944ca787b475294f9c41f6052a7ea P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.299131 26908 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b2c944ca787b475294f9c41f6052a7ea with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:02.300153 26921 catalog_manager.cc:5484] T b2c944ca787b475294f9c41f6052a7ea P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:02.302009 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet b2c944ca787b475294f9c41f6052a7ea with delete_type TABLET_DATA_TOMBSTONED (TS e5057264da53436ea17bad01c66a7fbf not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41962
I20240501 19:57:02.302208 904 tablet_replica.cc:330] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf: stopping tablet replica
I20240501 19:57:02.302345 904 raft_consensus.cc:2229] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:02.302534 904 raft_consensus.cc:2258] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:02.302958 673 raft_consensus.cc:1050] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e: attempting to promote NON_VOTER 045b41d2fb5545fd9975510efab305aa to VOTER
I20240501 19:57:02.303239 904 ts_tablet_manager.cc:1875] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:02.303267 673 consensus_queue.cc:237] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:02.304600 28876 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.304602 30036 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.304636 27459 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.305049 32684 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.305189 948 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.305277 904 ts_tablet_manager.cc:1888] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:02.305330 904 log.cc:1196] T b2c944ca787b475294f9c41f6052a7ea P e5057264da53436ea17bad01c66a7fbf: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/b2c944ca787b475294f9c41f6052a7ea
I20240501 19:57:02.305399 673 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.305778 26911 catalog_manager.cc:4830] TS e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115): tablet b2c944ca787b475294f9c41f6052a7ea (table rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]) successfully deleted
I20240501 19:57:02.307243 673 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.307664 28876 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.309437 27459 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.309551 26915 catalog_manager.cc:5484] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea reported cstate change: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.310218 30036 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.313505 27066 consensus_queue.cc:237] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:02.314815 30036 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.315152 948 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.316551 673 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.316924 30036 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.317879 26909 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 37288cafed764ba18fcc07b6e3c7a722 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:02.318570 28876 raft_consensus.cc:1261] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.318675 673 raft_consensus.cc:1050] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e: attempting to promote NON_VOTER 724bae28280242c6982c4f8e315b0d0b to VOTER
I20240501 19:57:02.318949 32684 consensus_queue.cc:1035] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.319057 673 consensus_queue.cc:237] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:57:02.319257 26921 catalog_manager.cc:5484] T 37288cafed764ba18fcc07b6e3c7a722 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:02.319816 28876 raft_consensus.cc:2932] T 37288cafed764ba18fcc07b6e3c7a722 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.321092 28877 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.321136 28746 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b [term 1 LEARNER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.321329 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet 37288cafed764ba18fcc07b6e3c7a722 with delete_type TABLET_DATA_TOMBSTONED (TS e5057264da53436ea17bad01c66a7fbf not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41962
I20240501 19:57:02.321511 904 tablet_replica.cc:330] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf: stopping tablet replica
I20240501 19:57:02.321527 27459 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.321647 904 raft_consensus.cc:2229] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:02.321789 904 raft_consensus.cc:2258] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:02.321929 32684 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.322072 948 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.322355 32684 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.323153 691 raft_consensus.cc:979] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: : Instructing follower 474eb4491dcb47f4a2a2f6e149bc39c1 to start an election
I20240501 19:57:02.323244 691 raft_consensus.cc:1067] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Signalling peer 474eb4491dcb47f4a2a2f6e149bc39c1 to start an election
I20240501 19:57:02.324095 29139 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0aaa59df03844aadafb8d57797685dc2"
dest_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1"
from {username='slave'} at 127.20.210.3:56397
I20240501 19:57:02.324249 29139 raft_consensus.cc:482] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 19:57:02.324314 29139 raft_consensus.cc:3037] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.325032 29139 raft_consensus.cc:504] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:57:02.325608 29139 leader_election.cc:288] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [CANDIDATE]: Term 2 election: Requested vote from peers e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115), 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155), 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529)
I20240501 19:57:02.325887 27328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aaa59df03844aadafb8d57797685dc2" candidate_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "3c3f70fdad7b438a841f74fc638d17f5"
I20240501 19:57:02.325992 27328 raft_consensus.cc:3032] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 19:57:02.326035 27328 raft_consensus.cc:729] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3f70fdad7b438a841f74fc638d17f5, State: Running, Role: LEADER
I20240501 19:57:02.326171 27328 consensus_queue.cc:260] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:57:02.326390 27328 raft_consensus.cc:3037] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.326855 27459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aaa59df03844aadafb8d57797685dc2" candidate_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
I20240501 19:57:02.326951 27459 raft_consensus.cc:3037] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.327359 27328 raft_consensus.cc:2454] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 474eb4491dcb47f4a2a2f6e149bc39c1 in term 2.
I20240501 19:57:02.327903 27459 raft_consensus.cc:2454] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 474eb4491dcb47f4a2a2f6e149bc39c1 in term 2.
I20240501 19:57:02.328224 29076 leader_election.cc:302] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c3f70fdad7b438a841f74fc638d17f5, 474eb4491dcb47f4a2a2f6e149bc39c1, e5057264da53436ea17bad01c66a7fbf; no voters:
I20240501 19:57:02.328392 933 raft_consensus.cc:2781] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 19:57:02.328480 933 raft_consensus.cc:686] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 2 LEADER]: Becoming Leader. State: Replica: 474eb4491dcb47f4a2a2f6e149bc39c1, State: Running, Role: LEADER
I20240501 19:57:02.328614 933 consensus_queue.cc:237] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:57:02.328594 950 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.330451 26921 catalog_manager.cc:5484] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: term changed from 1 to 2, leader changed from 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) to 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8). New cstate: current_term: 2 leader_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:02.330615 28877 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.331681 28746 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.333766 26916 catalog_manager.cc:5484] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:02.334789 904 ts_tablet_manager.cc:1875] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:02.336387 904 ts_tablet_manager.cc:1888] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:02.336441 904 log.cc:1196] T 37288cafed764ba18fcc07b6e3c7a722 P e5057264da53436ea17bad01c66a7fbf: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/37288cafed764ba18fcc07b6e3c7a722
I20240501 19:57:02.336941 26911 catalog_manager.cc:4830] TS e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115): tablet 37288cafed764ba18fcc07b6e3c7a722 (table rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]) successfully deleted
I20240501 19:57:02.337828 27066 consensus_queue.cc:237] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:57:02.338265 27459 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 724bae28280242c6982c4f8e315b0d0b (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.338922 28746 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.338959 28877 raft_consensus.cc:1261] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.339241 948 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.339386 673 consensus_queue.cc:1035] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.341629 673 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.342041 28746 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.342164 28877 raft_consensus.cc:2932] T 199f9529870242f599358995c19250b2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.343441 26909 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 199f9529870242f599358995c19250b2 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:02.344306 26921 catalog_manager.cc:5484] T 199f9529870242f599358995c19250b2 P 724bae28280242c6982c4f8e315b0d0b reported cstate change: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.346028 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet 199f9529870242f599358995c19250b2 with delete_type TABLET_DATA_TOMBSTONED (TS e5057264da53436ea17bad01c66a7fbf not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41962
I20240501 19:57:02.346225 904 tablet_replica.cc:330] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf: stopping tablet replica
I20240501 19:57:02.346374 904 raft_consensus.cc:2229] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:02.346549 904 raft_consensus.cc:2258] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:02.347174 904 ts_tablet_manager.cc:1875] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:02.348882 904 ts_tablet_manager.cc:1888] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:02.348943 904 log.cc:1196] T 199f9529870242f599358995c19250b2 P e5057264da53436ea17bad01c66a7fbf: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/199f9529870242f599358995c19250b2
I20240501 19:57:02.349364 26911 catalog_manager.cc:4830] TS e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115): tablet 199f9529870242f599358995c19250b2 (table rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]) successfully deleted
I20240501 19:57:02.349385 28877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aaa59df03844aadafb8d57797685dc2" candidate_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "7ccb4fe66ab942d399551b10f5ed32ea"
I20240501 19:57:02.349514 28877 raft_consensus.cc:3037] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.350459 28877 raft_consensus.cc:2454] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 474eb4491dcb47f4a2a2f6e149bc39c1 in term 2.
I20240501 19:57:02.351373 32684 raft_consensus.cc:1050] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e: attempting to promote NON_VOTER 045b41d2fb5545fd9975510efab305aa to VOTER
I20240501 19:57:02.351665 32684 consensus_queue.cc:237] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:02.352766 29139 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.353063 27196 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.353210 30036 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.353341 673 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.353484 948 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.353583 32684 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.355254 673 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.355667 27196 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.356891 29139 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.357662 26916 catalog_manager.cc:5484] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:02.358357 30036 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.361867 27066 consensus_queue.cc:237] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:02.362946 30036 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.363320 673 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.363982 29139 raft_consensus.cc:1261] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.364332 673 consensus_queue.cc:1035] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.365486 950 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.366550 26909 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c9f961f0554c45a1a3f31d5a8b31a46d with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:02.366612 30036 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.367430 26916 catalog_manager.cc:5484] T c9f961f0554c45a1a3f31d5a8b31a46d P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 8 to 9, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:02.368481 29139 raft_consensus.cc:2932] T c9f961f0554c45a1a3f31d5a8b31a46d P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.369763 27175 tablet_service.cc:1514] Processing DeleteTablet for tablet c9f961f0554c45a1a3f31d5a8b31a46d with delete_type TABLET_DATA_TOMBSTONED (TS ef2d33d636574e8fbb9d3b90a15035d6 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:43438
I20240501 19:57:02.370148 964 tablet_replica.cc:330] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6: stopping tablet replica
I20240501 19:57:02.370491 964 raft_consensus.cc:2229] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:02.370684 964 raft_consensus.cc:2258] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:02.371414 964 ts_tablet_manager.cc:1875] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:02.372944 964 ts_tablet_manager.cc:1888] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:02.373008 964 log.cc:1196] T c9f961f0554c45a1a3f31d5a8b31a46d P ef2d33d636574e8fbb9d3b90a15035d6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/c9f961f0554c45a1a3f31d5a8b31a46d
I20240501 19:57:02.373423 26908 catalog_manager.cc:4830] TS ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113): tablet c9f961f0554c45a1a3f31d5a8b31a46d (table rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]) successfully deleted
I20240501 19:57:02.462119 691 raft_consensus.cc:1050] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER ecdc0d100f5f4b95b9dda10c2f97cf20 to VOTER
I20240501 19:57:02.462513 691 consensus_queue.cc:237] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:57:02.463973 29008 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.464079 27066 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.464504 966 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.464710 689 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.465591 28877 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.466096 32689 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.467021 689 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:02.467447 27066 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:02.468740 28877 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:02.469666 29008 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:02.471539 26916 catalog_manager.cc:5484] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea reported cstate change: config changed from index 7 to 8, ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:02.475028 27328 consensus_queue.cc:237] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } }
I20240501 19:57:02.476158 29008 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.476351 28877 raft_consensus.cc:1261] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.476502 966 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.477036 966 consensus_queue.cc:1035] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.478128 32689 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:02.478847 28877 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:02.480086 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3e05a10bfc6540c19faa154896b8976a with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:02.481118 29008 raft_consensus.cc:2932] T 3e05a10bfc6540c19faa154896b8976a P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } }
I20240501 19:57:02.481524 26924 catalog_manager.cc:5484] T 3e05a10bfc6540c19faa154896b8976a P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 8 to 9, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:02.483920 27045 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e05a10bfc6540c19faa154896b8976a with delete_type TABLET_DATA_TOMBSTONED (TS 16cfc5a5082742c5b09d35e5c4f7f10e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:49644
I20240501 19:57:02.484303 985 tablet_replica.cc:330] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e: stopping tablet replica
I20240501 19:57:02.484516 985 raft_consensus.cc:2229] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:02.484865 985 raft_consensus.cc:2258] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:02.485641 985 ts_tablet_manager.cc:1875] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:02.488390 985 ts_tablet_manager.cc:1888] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:02.494385 985 log.cc:1196] T 3e05a10bfc6540c19faa154896b8976a P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/3e05a10bfc6540c19faa154896b8976a
I20240501 19:57:02.494868 26909 catalog_manager.cc:4830] TS 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375): tablet 3e05a10bfc6540c19faa154896b8976a (table rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]) successfully deleted
I20240501 19:57:02.507001 27196 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "55d39ed3702046b08b1ac3c76c272c01"
dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:50372
I20240501 19:57:02.507117 27196 raft_consensus.cc:595] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:57:02.519809 853 raft_consensus.cc:1050] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6: attempting to promote NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea to VOTER
I20240501 19:57:02.520251 853 consensus_queue.cc:237] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:57:02.521459 28877 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 LEARNER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.521768 28746 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.522118 850 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.522162 27459 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.522634 853 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.522992 853 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.525151 32691 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:02.525998 28746 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:02.526405 27459 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:02.527660 28877 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:02.529310 26918 catalog_manager.cc:5484] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 reported cstate change: config changed from index 7 to 8, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:02.532013 27196 consensus_queue.cc:237] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:57:02.533057 28877 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.533128 28746 raft_consensus.cc:1261] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer ef2d33d636574e8fbb9d3b90a15035d6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.533406 32691 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.533555 851 consensus_queue.cc:1035] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.535323 853 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:02.535650 28877 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:02.536684 26908 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0f417f4001bc48158187ed262f7737ab with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:02.536883 28746 raft_consensus.cc:2932] T 0f417f4001bc48158187ed262f7737ab P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:02.538352 26921 catalog_manager.cc:5484] T 0f417f4001bc48158187ed262f7737ab P 7ccb4fe66ab942d399551b10f5ed32ea reported cstate change: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:02.540745 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f417f4001bc48158187ed262f7737ab with delete_type TABLET_DATA_TOMBSTONED (TS e5057264da53436ea17bad01c66a7fbf not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41962
I20240501 19:57:02.540966 904 tablet_replica.cc:330] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf: stopping tablet replica
I20240501 19:57:02.541124 904 raft_consensus.cc:2229] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:02.541307 904 raft_consensus.cc:2258] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:02.542145 904 ts_tablet_manager.cc:1875] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:02.542573 604 raft_consensus.cc:1050] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf: attempting to promote NON_VOTER 045b41d2fb5545fd9975510efab305aa to VOTER
I20240501 19:57:02.542867 604 consensus_queue.cc:237] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:02.544013 904 ts_tablet_manager.cc:1888] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:02.544064 30036 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.544092 904 log.cc:1196] T 0f417f4001bc48158187ed262f7737ab P e5057264da53436ea17bad01c66a7fbf: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/0f417f4001bc48158187ed262f7737ab
I20240501 19:57:02.544445 1031 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.544579 26911 catalog_manager.cc:4830] TS e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115): tablet 0f417f4001bc48158187ed262f7737ab (table rebalance_test_table_1 [id=e02b2578e7c44c89a95a6964d3b647ce]) successfully deleted
I20240501 19:57:02.545002 27328 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.545025 29139 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:02.545408 1034 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.545701 1031 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.547868 1031 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.548489 30036 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.549135 29140 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.550542 26924 catalog_manager.cc:5484] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:02.551007 27328 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.553450 27459 consensus_queue.cc:237] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:02.554587 29140 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.554589 30036 raft_consensus.cc:1261] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:02.554925 1033 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.555073 1034 consensus_queue.cc:1035] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.557024 1033 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.557615 29140 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.558846 30036 raft_consensus.cc:2932] T 9102f7c8350843c2bd7009ea190860d0 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.558944 26911 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9102f7c8350843c2bd7009ea190860d0 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:02.560072 26924 catalog_manager.cc:5484] T 9102f7c8350843c2bd7009ea190860d0 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: config changed from index 8 to 9, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:02.562063 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet 9102f7c8350843c2bd7009ea190860d0 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3f70fdad7b438a841f74fc638d17f5 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:34348
I20240501 19:57:02.562462 1043 tablet_replica.cc:330] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5: stopping tablet replica
I20240501 19:57:02.562602 1043 raft_consensus.cc:2229] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:02.562786 1043 raft_consensus.cc:2258] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:02.563397 1043 ts_tablet_manager.cc:1875] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:02.565191 1043 ts_tablet_manager.cc:1888] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:02.565254 1043 log.cc:1196] T 9102f7c8350843c2bd7009ea190860d0 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/9102f7c8350843c2bd7009ea190860d0
I20240501 19:57:02.565727 26910 catalog_manager.cc:4830] TS 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155): tablet 9102f7c8350843c2bd7009ea190860d0 (table rebalance_test_table_4 [id=4ae5acbb5bfd481289bbef767f63d182]) successfully deleted
I20240501 19:57:02.573407 27066 consensus_queue.cc:237] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:02.574738 27459 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
W20240501 19:57:02.574859 27002 consensus_peers.cc:485] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e -> Peer 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833): Couldn't send request to peer 045b41d2fb5545fd9975510efab305aa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 692732b291b441029826cf20c9a0ecb1. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:02.574911 29008 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240501 19:57:02.575093 950 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.575569 950 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:02.577271 948 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:02.577688 27459 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:02.577857 29008 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:02.580286 26916 catalog_manager.cc:5484] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 reported cstate change: config changed from index 7 to 8, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New cstate: current_term: 2 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:02.587782 27328 consensus_queue.cc:237] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:02.589274 28877 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:02.589351 27459 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:02.589630 966 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240501 19:57:02.589784 27264 consensus_peers.cc:485] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833): Couldn't send request to peer 045b41d2fb5545fd9975510efab305aa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0723850372f8416d9b8ac19f119939bc. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:02.589892 32689 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:02.591367 689 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:02.591789 27459 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:02.593778 27196 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:02.594179 26924 catalog_manager.cc:5484] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:02.594784 874 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.594957 28877 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:02.611091 851 raft_consensus.cc:979] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: : Instructing follower ecdc0d100f5f4b95b9dda10c2f97cf20 to start an election
I20240501 19:57:02.611189 851 raft_consensus.cc:1067] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Signalling peer ecdc0d100f5f4b95b9dda10c2f97cf20 to start an election
I20240501 19:57:02.611595 29008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "55d39ed3702046b08b1ac3c76c272c01"
dest_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20"
from {username='slave'} at 127.20.210.2:57789
I20240501 19:57:02.611719 29008 raft_consensus.cc:482] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 19:57:02.611804 29008 raft_consensus.cc:3037] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.612551 29008 raft_consensus.cc:504] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:02.612982 29008 leader_election.cc:288] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [CANDIDATE]: Term 2 election: Requested vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113), 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833)
I20240501 19:57:02.613210 27196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d39ed3702046b08b1ac3c76c272c01" candidate_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "ef2d33d636574e8fbb9d3b90a15035d6"
I20240501 19:57:02.613315 27196 raft_consensus.cc:3032] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 19:57:02.613369 27196 raft_consensus.cc:729] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ef2d33d636574e8fbb9d3b90a15035d6, State: Running, Role: LEADER
I20240501 19:57:02.613504 27196 consensus_queue.cc:260] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:02.613675 27196 raft_consensus.cc:3037] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.614506 27196 raft_consensus.cc:2454] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecdc0d100f5f4b95b9dda10c2f97cf20 in term 2.
I20240501 19:57:02.614754 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d39ed3702046b08b1ac3c76c272c01" candidate_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:57:02.614868 27066 raft_consensus.cc:3037] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.615909 27066 raft_consensus.cc:2454] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecdc0d100f5f4b95b9dda10c2f97cf20 in term 2.
I20240501 19:57:02.616181 28942 leader_election.cc:302] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 16cfc5a5082742c5b09d35e5c4f7f10e, ecdc0d100f5f4b95b9dda10c2f97cf20, ef2d33d636574e8fbb9d3b90a15035d6; no voters:
I20240501 19:57:02.616379 874 raft_consensus.cc:2781] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 19:57:02.616462 874 raft_consensus.cc:686] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 LEADER]: Becoming Leader. State: Replica: ecdc0d100f5f4b95b9dda10c2f97cf20, State: Running, Role: LEADER
I20240501 19:57:02.616598 874 consensus_queue.cc:237] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:02.616773 30036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d39ed3702046b08b1ac3c76c272c01" candidate_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "045b41d2fb5545fd9975510efab305aa"
I20240501 19:57:02.616920 30036 raft_consensus.cc:3037] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:02.617816 30036 raft_consensus.cc:2454] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecdc0d100f5f4b95b9dda10c2f97cf20 in term 2.
I20240501 19:57:02.618201 26924 catalog_manager.cc:5484] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 reported cstate change: term changed from 1 to 2, leader changed from ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) to ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7). New cstate: current_term: 2 leader_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:02.639503 27196 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:02.639962 512 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.645521 27328 consensus_queue.cc:237] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:02.646859 27459 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:02.646986 28746 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:02.647250 966 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:02.647423 691 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240501 19:57:02.647749 27264 consensus_peers.cc:485] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 -> Peer 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833): Couldn't send request to peer 045b41d2fb5545fd9975510efab305aa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 75cc96baf4654318a09dcc34eb32819c. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:02.649091 32689 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:02.649497 27459 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:02.651099 28746 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:02.651301 26924 catalog_manager.cc:5484] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:02.663832 28746 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:02.664270 874 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.667732 27066 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:02.668265 1130 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.670548 29008 consensus_queue.cc:237] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } }
I20240501 19:57:02.671944 27196 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:02.672158 28746 raft_consensus.cc:1261] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:02.672595 1130 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:02.672853 874 consensus_queue.cc:1035] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:02.674333 30462 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.674736 28746 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.675751 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6c498eacc5e947bfb01dfb90c7039df0 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:02.676735 27196 raft_consensus.cc:2932] T 6c498eacc5e947bfb01dfb90c7039df0 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.677103 26921 catalog_manager.cc:5484] T 6c498eacc5e947bfb01dfb90c7039df0 P 724bae28280242c6982c4f8e315b0d0b reported cstate change: config changed from index 8 to 10, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New cstate: current_term: 2 leader_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } }
I20240501 19:57:02.679477 27045 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c498eacc5e947bfb01dfb90c7039df0 with delete_type TABLET_DATA_TOMBSTONED (TS 16cfc5a5082742c5b09d35e5c4f7f10e not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:49644
I20240501 19:57:02.679714 985 tablet_replica.cc:330] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: stopping tablet replica
I20240501 19:57:02.679863 985 raft_consensus.cc:2229] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:02.679966 985 raft_consensus.cc:2258] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:02.680651 985 ts_tablet_manager.cc:1875] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:02.682288 985 ts_tablet_manager.cc:1888] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 19:57:02.682354 985 log.cc:1196] T 6c498eacc5e947bfb01dfb90c7039df0 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/6c498eacc5e947bfb01dfb90c7039df0
I20240501 19:57:02.682818 26909 catalog_manager.cc:4830] TS 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375): tablet 6c498eacc5e947bfb01dfb90c7039df0 (table rebalance_test_table_2 [id=f4b0cf67feb647df87e1f8b2bcefcd62]) successfully deleted
I20240501 19:57:02.720670 27328 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:02.721038 512 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.723625 30036 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:02.724092 512 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.857642 27459 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:02.858029 512 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.864889 27328 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:02.865237 512 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:02.867568 28877 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:02.868032 933 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:03.030042 1223 ts_tablet_manager.cc:902] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:57:03.030565 1223 tablet_copy_client.cc:308] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:57:03.031054 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 75cc96baf4654318a09dcc34eb32819c from peer 045b41d2fb5545fd9975510efab305aa ({username='slave'} at 127.20.210.9:37755)
I20240501 19:57:03.031147 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 75cc96baf4654318a09dcc34eb32819c from peer 045b41d2fb5545fd9975510efab305aa at {username='slave'} at 127.20.210.9:37755: session id = 045b41d2fb5545fd9975510efab305aa-75cc96baf4654318a09dcc34eb32819c
I20240501 19:57:03.031858 27348 tablet_copy_source_session.cc:205] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:03.032598 1223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75cc96baf4654318a09dcc34eb32819c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:03.034219 1223 tablet_copy_client.cc:783] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:03.034391 1223 tablet_copy_client.cc:647] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:03.035466 1227 tablet_copy_client.cc:1002] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa: tablet copy: Injecting latency into file download: 1500
I20240501 19:57:03.106413 27196 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:03.106817 1130 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:03.107360 1228 ts_tablet_manager.cc:902] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa: Initiating tablet copy from peer 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:57:03.107934 1228 tablet_copy_client.cc:308] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:33375
I20240501 19:57:03.108412 27085 tablet_copy_service.cc:140] P 16cfc5a5082742c5b09d35e5c4f7f10e: Received BeginTabletCopySession request for tablet 692732b291b441029826cf20c9a0ecb1 from peer 045b41d2fb5545fd9975510efab305aa ({username='slave'} at 127.20.210.9:56871)
I20240501 19:57:03.108520 27085 tablet_copy_service.cc:161] P 16cfc5a5082742c5b09d35e5c4f7f10e: Beginning new tablet copy session on tablet 692732b291b441029826cf20c9a0ecb1 from peer 045b41d2fb5545fd9975510efab305aa at {username='slave'} at 127.20.210.9:56871: session id = 045b41d2fb5545fd9975510efab305aa-692732b291b441029826cf20c9a0ecb1
I20240501 19:57:03.109429 27085 tablet_copy_source_session.cc:205] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:03.110271 1228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 692732b291b441029826cf20c9a0ecb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:03.111793 1228 tablet_copy_client.cc:783] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:03.111940 1228 tablet_copy_client.cc:647] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:03.112828 1232 tablet_copy_client.cc:1002] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Injecting latency into file download: 1500
I20240501 19:57:03.125197 1233 ts_tablet_manager.cc:902] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa: Initiating tablet copy from peer 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155)
I20240501 19:57:03.125629 1233 tablet_copy_client.cc:308] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:46155
I20240501 19:57:03.126050 27348 tablet_copy_service.cc:140] P 3c3f70fdad7b438a841f74fc638d17f5: Received BeginTabletCopySession request for tablet 0723850372f8416d9b8ac19f119939bc from peer 045b41d2fb5545fd9975510efab305aa ({username='slave'} at 127.20.210.9:37755)
I20240501 19:57:03.126166 27348 tablet_copy_service.cc:161] P 3c3f70fdad7b438a841f74fc638d17f5: Beginning new tablet copy session on tablet 0723850372f8416d9b8ac19f119939bc from peer 045b41d2fb5545fd9975510efab305aa at {username='slave'} at 127.20.210.9:37755: session id = 045b41d2fb5545fd9975510efab305aa-0723850372f8416d9b8ac19f119939bc
I20240501 19:57:03.126348 27066 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:03.126724 1130 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:03.127106 27348 tablet_copy_source_session.cc:205] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:03.127988 1233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0723850372f8416d9b8ac19f119939bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:03.129124 30036 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:03.129638 1130 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:03.130007 1233 tablet_copy_client.cc:783] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:03.130162 1233 tablet_copy_client.cc:647] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:03.132004 1237 tablet_copy_client.cc:1002] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa: tablet copy: Injecting latency into file download: 1500
I20240501 19:57:03.147572 29140 consensus_queue.cc:237] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:03.148543 30036 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:03.148758 27196 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:03.149083 933 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:03.149470 1186 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:03.153854 1186 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:03.155953 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5a9674523a2148b1a6c1a5d22a57aac5 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:03.155903 30036 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:03.157618 27196 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:03.157964 26921 catalog_manager.cc:5484] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: config changed from index 8 to 10, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New cstate: current_term: 2 leader_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:03.160434 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a9674523a2148b1a6c1a5d22a57aac5 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3f70fdad7b438a841f74fc638d17f5 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:34348
I20240501 19:57:03.160806 1241 tablet_replica.cc:330] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: stopping tablet replica
I20240501 19:57:03.160959 1241 raft_consensus.cc:2229] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:03.161067 1241 raft_consensus.cc:2258] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:03.161775 1241 ts_tablet_manager.cc:1875] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:03.164036 1241 ts_tablet_manager.cc:1888] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 19:57:03.164116 1241 log.cc:1196] T 5a9674523a2148b1a6c1a5d22a57aac5 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/5a9674523a2148b1a6c1a5d22a57aac5
I20240501 19:57:03.164618 26910 catalog_manager.cc:4830] TS 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155): tablet 5a9674523a2148b1a6c1a5d22a57aac5 (table rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]) successfully deleted
I20240501 19:57:03.200769 27066 consensus_queue.cc:237] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:03.202152 27328 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:03.202219 29140 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240501 19:57:03.202433 27002 consensus_peers.cc:485] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e -> Peer 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833): Couldn't send request to peer 045b41d2fb5545fd9975510efab305aa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6aff1d7fd2d2476095ef0a0529110b49. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:03.202687 950 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:03.202880 32684 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:03.204645 948 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:03.205096 29140 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:03.205122 27328 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:03.207829 26916 catalog_manager.cc:5484] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:03.362814 29140 consensus_queue.cc:237] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } }
I20240501 19:57:03.363868 28876 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:03.363957 27459 raft_consensus.cc:1261] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:03.364226 512 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:03.364387 933 consensus_queue.cc:1035] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:03.365972 933 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:57:03.367084 28876 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P 7ccb4fe66ab942d399551b10f5ed32ea [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:57:03.368837 27459 raft_consensus.cc:2932] T 0aaa59df03844aadafb8d57797685dc2 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } }
I20240501 19:57:03.369414 26924 catalog_manager.cc:5484] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: config changed from index 8 to 10, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New cstate: current_term: 2 leader_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:03.370692 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0aaa59df03844aadafb8d57797685dc2 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:03.372125 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aaa59df03844aadafb8d57797685dc2 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3f70fdad7b438a841f74fc638d17f5 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:34348
I20240501 19:57:03.372323 1241 tablet_replica.cc:330] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: stopping tablet replica
I20240501 19:57:03.372444 1241 raft_consensus.cc:2229] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:03.372542 1241 raft_consensus.cc:2258] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:03.373081 1241 ts_tablet_manager.cc:1875] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:03.374636 1241 ts_tablet_manager.cc:1888] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 19:57:03.374696 1241 log.cc:1196] T 0aaa59df03844aadafb8d57797685dc2 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/0aaa59df03844aadafb8d57797685dc2
I20240501 19:57:03.375084 26910 catalog_manager.cc:4830] TS 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155): tablet 0aaa59df03844aadafb8d57797685dc2 (table rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]) successfully deleted
I20240501 19:57:03.519876 29008 consensus_queue.cc:237] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:03.520898 27066 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:03.520946 30036 raft_consensus.cc:1261] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Refusing update from remote peer ecdc0d100f5f4b95b9dda10c2f97cf20: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:03.521261 874 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:03.521420 1130 consensus_queue.cc:1035] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:03.523072 30462 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:03.523672 27066 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:03.524770 30036 raft_consensus.cc:2932] T 55d39ed3702046b08b1ac3c76c272c01 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:03.525663 26921 catalog_manager.cc:5484] T 55d39ed3702046b08b1ac3c76c272c01 P ecdc0d100f5f4b95b9dda10c2f97cf20 reported cstate change: config changed from index 8 to 10, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New cstate: current_term: 2 leader_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:03.525938 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 55d39ed3702046b08b1ac3c76c272c01 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:03.528084 27175 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d39ed3702046b08b1ac3c76c272c01 with delete_type TABLET_DATA_TOMBSTONED (TS ef2d33d636574e8fbb9d3b90a15035d6 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:43438
I20240501 19:57:03.528591 1367 tablet_replica.cc:330] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: stopping tablet replica
I20240501 19:57:03.528762 1367 raft_consensus.cc:2229] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:03.528878 1367 raft_consensus.cc:2258] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:03.529451 1367 ts_tablet_manager.cc:1875] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:03.531196 1367 ts_tablet_manager.cc:1888] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240501 19:57:03.531261 1367 log.cc:1196] T 55d39ed3702046b08b1ac3c76c272c01 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/55d39ed3702046b08b1ac3c76c272c01
I20240501 19:57:03.531682 26908 catalog_manager.cc:4830] TS ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113): tablet 55d39ed3702046b08b1ac3c76c272c01 (table rebalance_test_table_9 [id=c49992309be645c29102d57f864565fe]) successfully deleted
I20240501 19:57:03.737401 27459 consensus_queue.cc:237] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
W20240501 19:57:03.738981 27395 consensus_peers.cc:485] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf -> Peer 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833): Couldn't send request to peer 045b41d2fb5545fd9975510efab305aa. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6ee4db743833421e8120edbec4ba0836. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:03.738996 27328 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:03.738996 29140 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:03.739328 1400 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:03.739526 1401 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:03.741132 1031 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:03.741596 29140 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:03.741587 27328 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:03.743959 26918 catalog_manager.cc:5484] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: config changed from index 6 to 7, NON_VOTER 045b41d2fb5545fd9975510efab305aa (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } } }
I20240501 19:57:03.752820 1414 ts_tablet_manager.cc:902] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa: Initiating tablet copy from peer 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375)
I20240501 19:57:03.753347 1414 tablet_copy_client.cc:308] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:33375
I20240501 19:57:03.753914 27085 tablet_copy_service.cc:140] P 16cfc5a5082742c5b09d35e5c4f7f10e: Received BeginTabletCopySession request for tablet 6aff1d7fd2d2476095ef0a0529110b49 from peer 045b41d2fb5545fd9975510efab305aa ({username='slave'} at 127.20.210.9:56871)
I20240501 19:57:03.754001 27085 tablet_copy_service.cc:161] P 16cfc5a5082742c5b09d35e5c4f7f10e: Beginning new tablet copy session on tablet 6aff1d7fd2d2476095ef0a0529110b49 from peer 045b41d2fb5545fd9975510efab305aa at {username='slave'} at 127.20.210.9:56871: session id = 045b41d2fb5545fd9975510efab305aa-6aff1d7fd2d2476095ef0a0529110b49
I20240501 19:57:03.754793 27085 tablet_copy_source_session.cc:205] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:03.756160 1414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aff1d7fd2d2476095ef0a0529110b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:03.758129 1414 tablet_copy_client.cc:783] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:03.758291 1414 tablet_copy_client.cc:647] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:03.759557 1415 tablet_copy_client.cc:1002] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Injecting latency into file download: 1500
I20240501 19:57:04.275034 1485 ts_tablet_manager.cc:902] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa: Initiating tablet copy from peer e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115)
I20240501 19:57:04.275487 1485 tablet_copy_client.cc:308] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:37115
I20240501 19:57:04.275949 27479 tablet_copy_service.cc:140] P e5057264da53436ea17bad01c66a7fbf: Received BeginTabletCopySession request for tablet 6ee4db743833421e8120edbec4ba0836 from peer 045b41d2fb5545fd9975510efab305aa ({username='slave'} at 127.20.210.9:36467)
I20240501 19:57:04.276042 27479 tablet_copy_service.cc:161] P e5057264da53436ea17bad01c66a7fbf: Beginning new tablet copy session on tablet 6ee4db743833421e8120edbec4ba0836 from peer 045b41d2fb5545fd9975510efab305aa at {username='slave'} at 127.20.210.9:36467: session id = 045b41d2fb5545fd9975510efab305aa-6ee4db743833421e8120edbec4ba0836
I20240501 19:57:04.276724 27479 tablet_copy_source_session.cc:205] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:04.277580 1485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee4db743833421e8120edbec4ba0836. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:04.279249 1485 tablet_copy_client.cc:783] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:04.279389 1485 tablet_copy_client.cc:647] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:04.280282 1489 tablet_copy_client.cc:1002] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Injecting latency into file download: 1500
I20240501 19:57:04.535722 1223 tablet_copy_client.cc:523] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:04.537163 1223 tablet_bootstrap.cc:492] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa: Bootstrap starting.
I20240501 19:57:04.553941 1223 tablet_bootstrap.cc:492] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:04.554134 1223 tablet_bootstrap.cc:492] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa: Bootstrap complete.
I20240501 19:57:04.554288 1223 ts_tablet_manager.cc:1367] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa: Time spent bootstrapping tablet: real 0.017s user 0.017s sys 0.002s
I20240501 19:57:04.554620 1223 raft_consensus.cc:348] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:04.554711 1223 raft_consensus.cc:729] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 045b41d2fb5545fd9975510efab305aa, State: Initialized, Role: LEARNER
I20240501 19:57:04.554790 1223 consensus_queue.cc:260] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:04.555027 1223 ts_tablet_manager.cc:1398] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:04.555445 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session 045b41d2fb5545fd9975510efab305aa-75cc96baf4654318a09dcc34eb32819c received from {username='slave'} at 127.20.210.9:37755
I20240501 19:57:04.555534 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session 045b41d2fb5545fd9975510efab305aa-75cc96baf4654318a09dcc34eb32819c on tablet 75cc96baf4654318a09dcc34eb32819c with peer 045b41d2fb5545fd9975510efab305aa
I20240501 19:57:04.613107 1228 tablet_copy_client.cc:523] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:04.614343 1228 tablet_bootstrap.cc:492] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa: Bootstrap starting.
I20240501 19:57:04.632263 1233 tablet_copy_client.cc:523] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:04.632465 1228 tablet_bootstrap.cc:492] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:04.632714 1228 tablet_bootstrap.cc:492] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa: Bootstrap complete.
I20240501 19:57:04.632922 1228 ts_tablet_manager.cc:1367] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.003s
I20240501 19:57:04.633273 1228 raft_consensus.cc:348] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:04.633399 1228 raft_consensus.cc:729] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 045b41d2fb5545fd9975510efab305aa, State: Initialized, Role: LEARNER
I20240501 19:57:04.633455 1233 tablet_bootstrap.cc:492] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa: Bootstrap starting.
I20240501 19:57:04.633514 1228 consensus_queue.cc:260] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:04.633824 1228 ts_tablet_manager.cc:1398] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:04.634305 27085 tablet_copy_service.cc:342] P 16cfc5a5082742c5b09d35e5c4f7f10e: Request end of tablet copy session 045b41d2fb5545fd9975510efab305aa-692732b291b441029826cf20c9a0ecb1 received from {username='slave'} at 127.20.210.9:56871
I20240501 19:57:04.634392 27085 tablet_copy_service.cc:433] P 16cfc5a5082742c5b09d35e5c4f7f10e: ending tablet copy session 045b41d2fb5545fd9975510efab305aa-692732b291b441029826cf20c9a0ecb1 on tablet 692732b291b441029826cf20c9a0ecb1 with peer 045b41d2fb5545fd9975510efab305aa
I20240501 19:57:04.651763 1233 tablet_bootstrap.cc:492] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:04.652005 1233 tablet_bootstrap.cc:492] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa: Bootstrap complete.
I20240501 19:57:04.652154 1233 ts_tablet_manager.cc:1367] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa: Time spent bootstrapping tablet: real 0.019s user 0.018s sys 0.000s
I20240501 19:57:04.652578 1233 raft_consensus.cc:348] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:04.652678 1233 raft_consensus.cc:729] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 045b41d2fb5545fd9975510efab305aa, State: Initialized, Role: LEARNER
I20240501 19:57:04.652760 1233 consensus_queue.cc:260] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:04.653034 1233 ts_tablet_manager.cc:1398] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:04.653564 27348 tablet_copy_service.cc:342] P 3c3f70fdad7b438a841f74fc638d17f5: Request end of tablet copy session 045b41d2fb5545fd9975510efab305aa-0723850372f8416d9b8ac19f119939bc received from {username='slave'} at 127.20.210.9:37755
I20240501 19:57:04.653649 27348 tablet_copy_service.cc:433] P 3c3f70fdad7b438a841f74fc638d17f5: ending tablet copy session 045b41d2fb5545fd9975510efab305aa-0723850372f8416d9b8ac19f119939bc on tablet 0723850372f8416d9b8ac19f119939bc with peer 045b41d2fb5545fd9975510efab305aa
I20240501 19:57:04.902755 30036 raft_consensus.cc:1203] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:05.099114 30036 raft_consensus.cc:1203] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:05.186625 30036 raft_consensus.cc:1203] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa [term 2 LEARNER]: Deduplicated request from leader. Original: 2.7->[2.8-2.8] Dedup: 2.8->[]
I20240501 19:57:05.259843 1414 tablet_copy_client.cc:523] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:05.261065 1414 tablet_bootstrap.cc:492] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa: Bootstrap starting.
I20240501 19:57:05.277199 1414 tablet_bootstrap.cc:492] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:05.277463 1414 tablet_bootstrap.cc:492] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa: Bootstrap complete.
I20240501 19:57:05.277626 1414 ts_tablet_manager.cc:1367] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.001s
I20240501 19:57:05.277994 1414 raft_consensus.cc:348] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:05.278108 1414 raft_consensus.cc:729] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 045b41d2fb5545fd9975510efab305aa, State: Initialized, Role: LEARNER
I20240501 19:57:05.278208 1414 consensus_queue.cc:260] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:05.278524 1414 ts_tablet_manager.cc:1398] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 19:57:05.279047 27085 tablet_copy_service.cc:342] P 16cfc5a5082742c5b09d35e5c4f7f10e: Request end of tablet copy session 045b41d2fb5545fd9975510efab305aa-6aff1d7fd2d2476095ef0a0529110b49 received from {username='slave'} at 127.20.210.9:56871
I20240501 19:57:05.279129 27085 tablet_copy_service.cc:433] P 16cfc5a5082742c5b09d35e5c4f7f10e: ending tablet copy session 045b41d2fb5545fd9975510efab305aa-6aff1d7fd2d2476095ef0a0529110b49 on tablet 6aff1d7fd2d2476095ef0a0529110b49 with peer 045b41d2fb5545fd9975510efab305aa
I20240501 19:57:05.342098 1608 raft_consensus.cc:1050] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER 045b41d2fb5545fd9975510efab305aa to VOTER
I20240501 19:57:05.342517 1608 consensus_queue.cc:237] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:05.343904 27458 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:05.344445 28745 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:05.344511 30036 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:05.344732 1602 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:05.344944 1608 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:05.345538 1608 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:05.347316 1602 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.347721 30036 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.349627 28745 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.350792 26916 catalog_manager.cc:5484] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa reported cstate change: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.353127 27458 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.357482 27328 consensus_queue.cc:237] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:05.358628 30036 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:05.358660 28745 raft_consensus.cc:1261] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:05.359102 32689 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:05.359545 1602 consensus_queue.cc:1035] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:05.360854 32689 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.361196 30036 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.361732 28745 raft_consensus.cc:2932] T 75cc96baf4654318a09dcc34eb32819c P 724bae28280242c6982c4f8e315b0d0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.363171 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 75cc96baf4654318a09dcc34eb32819c with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:05.364248 26924 catalog_manager.cc:5484] T 75cc96baf4654318a09dcc34eb32819c P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "724bae28280242c6982c4f8e315b0d0b" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 42343 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:05.366796 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet 75cc96baf4654318a09dcc34eb32819c with delete_type TABLET_DATA_TOMBSTONED (TS e5057264da53436ea17bad01c66a7fbf not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41962
I20240501 19:57:05.367177 1635 tablet_replica.cc:330] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf: stopping tablet replica
I20240501 19:57:05.367329 1635 raft_consensus.cc:2229] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:05.367651 1635 raft_consensus.cc:2258] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:05.368373 1635 ts_tablet_manager.cc:1875] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:05.370586 1635 ts_tablet_manager.cc:1888] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:05.370668 1635 log.cc:1196] T 75cc96baf4654318a09dcc34eb32819c P e5057264da53436ea17bad01c66a7fbf: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/75cc96baf4654318a09dcc34eb32819c
I20240501 19:57:05.371208 26911 catalog_manager.cc:4830] TS e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115): tablet 75cc96baf4654318a09dcc34eb32819c (table rebalance_test_table_7 [id=c6098ad85d7a49c689039dc7379dfd1c]) successfully deleted
I20240501 19:57:05.475594 1608 raft_consensus.cc:1050] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5: attempting to promote NON_VOTER 045b41d2fb5545fd9975510efab305aa to VOTER
I20240501 19:57:05.476006 1608 consensus_queue.cc:237] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:05.477212 30036 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:05.477242 27458 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:05.477294 28877 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:05.477634 1641 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:05.477836 1608 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:05.478000 32689 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:05.479717 1608 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.480151 27458 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.481379 28877 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.481571 30036 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.482447 26918 catalog_manager.cc:5484] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 reported cstate change: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:05.485466 27326 consensus_queue.cc:237] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:05.486554 30036 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:05.486752 28877 raft_consensus.cc:1261] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Refusing update from remote peer 3c3f70fdad7b438a841f74fc638d17f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:05.486932 1608 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:05.487102 32689 consensus_queue.cc:1035] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:05.488493 32689 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.489662 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0723850372f8416d9b8ac19f119939bc with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:05.490135 30036 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.492002 26924 catalog_manager.cc:5484] T 0723850372f8416d9b8ac19f119939bc P 045b41d2fb5545fd9975510efab305aa reported cstate change: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "3c3f70fdad7b438a841f74fc638d17f5" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.493785 27437 tablet_service.cc:1514] Processing DeleteTablet for tablet 0723850372f8416d9b8ac19f119939bc with delete_type TABLET_DATA_TOMBSTONED (TS e5057264da53436ea17bad01c66a7fbf not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:41962
I20240501 19:57:05.493798 28877 raft_consensus.cc:2932] T 0723850372f8416d9b8ac19f119939bc P 7ccb4fe66ab942d399551b10f5ed32ea [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER e5057264da53436ea17bad01c66a7fbf (127.20.210.4) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.494027 1635 tablet_replica.cc:330] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf: stopping tablet replica
I20240501 19:57:05.494143 1635 raft_consensus.cc:2229] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:05.494323 1635 raft_consensus.cc:2258] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:05.495020 1635 ts_tablet_manager.cc:1875] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:05.496783 1635 ts_tablet_manager.cc:1888] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:05.496834 1635 log.cc:1196] T 0723850372f8416d9b8ac19f119939bc P e5057264da53436ea17bad01c66a7fbf: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/0723850372f8416d9b8ac19f119939bc
I20240501 19:57:05.497258 26911 catalog_manager.cc:4830] TS e5057264da53436ea17bad01c66a7fbf (127.20.210.4:37115): tablet 0723850372f8416d9b8ac19f119939bc (table rebalance_test_table_6 [id=81a4c5fcf3f94eeaa074f183a9ad44e6]) successfully deleted
I20240501 19:57:05.527783 30036 raft_consensus.cc:1203] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:05.621086 1687 raft_consensus.cc:1050] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e: attempting to promote NON_VOTER 045b41d2fb5545fd9975510efab305aa to VOTER
I20240501 19:57:05.621424 1687 consensus_queue.cc:237] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:05.622668 27458 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:05.622676 29008 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:05.623011 1687 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:05.623041 30036 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa [term 2 LEARNER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240501 19:57:05.623171 32684 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:05.623414 32684 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:05.625465 32684 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.625795 27458 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.625881 30036 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.628165 26924 catalog_manager.cc:5484] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 8 to 9, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:05.628741 29008 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.780572 1485 tablet_copy_client.cc:523] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:05.781764 1485 tablet_bootstrap.cc:492] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa: Bootstrap starting.
I20240501 19:57:05.798310 1485 tablet_bootstrap.cc:492] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:05.798555 1485 tablet_bootstrap.cc:492] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa: Bootstrap complete.
I20240501 19:57:05.798720 1485 ts_tablet_manager.cc:1367] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa: Time spent bootstrapping tablet: real 0.017s user 0.018s sys 0.000s
I20240501 19:57:05.799088 1485 raft_consensus.cc:348] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:05.799191 1485 raft_consensus.cc:729] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 045b41d2fb5545fd9975510efab305aa, State: Initialized, Role: LEARNER
I20240501 19:57:05.799281 1485 consensus_queue.cc:260] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: true } }
I20240501 19:57:05.799539 1485 ts_tablet_manager.cc:1398] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:05.800082 27479 tablet_copy_service.cc:342] P e5057264da53436ea17bad01c66a7fbf: Request end of tablet copy session 045b41d2fb5545fd9975510efab305aa-6ee4db743833421e8120edbec4ba0836 received from {username='slave'} at 127.20.210.9:36467
I20240501 19:57:05.800177 27479 tablet_copy_service.cc:433] P e5057264da53436ea17bad01c66a7fbf: ending tablet copy session 045b41d2fb5545fd9975510efab305aa-6ee4db743833421e8120edbec4ba0836 on tablet 6ee4db743833421e8120edbec4ba0836 with peer 045b41d2fb5545fd9975510efab305aa
I20240501 19:57:05.833104 27066 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1"
dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53002
I20240501 19:57:05.833231 27066 raft_consensus.cc:595] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 LEADER]: Received request to transfer leadership
I20240501 19:57:05.839553 30036 raft_consensus.cc:1203] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240501 19:57:05.942493 1689 raft_consensus.cc:1050] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e: attempting to promote NON_VOTER 045b41d2fb5545fd9975510efab305aa to VOTER
I20240501 19:57:05.942883 1689 consensus_queue.cc:237] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:05.944120 30036 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:05.944217 29140 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:05.944274 27326 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:05.944532 1689 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:05.944684 1517 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:05.944805 1687 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:05.946559 1687 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.947000 27326 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.947870 30036 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.949159 29140 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.949469 26916 catalog_manager.cc:5484] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e reported cstate change: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:05.952479 27066 consensus_queue.cc:237] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:05.953511 30036 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:05.953604 29140 raft_consensus.cc:1261] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer 16cfc5a5082742c5b09d35e5c4f7f10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:05.953904 1689 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:05.954077 1517 consensus_queue.cc:1035] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:05.955519 32684 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.955897 29140 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.956687 26909 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6aff1d7fd2d2476095ef0a0529110b49 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:05.957491 30036 raft_consensus.cc:2932] T 6aff1d7fd2d2476095ef0a0529110b49 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.957898 26921 catalog_manager.cc:5484] T 6aff1d7fd2d2476095ef0a0529110b49 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: config changed from index 8 to 9, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:05.960234 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aff1d7fd2d2476095ef0a0529110b49 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3f70fdad7b438a841f74fc638d17f5 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:34348
I20240501 19:57:05.960582 1718 tablet_replica.cc:330] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5: stopping tablet replica
I20240501 19:57:05.960685 1718 raft_consensus.cc:2229] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:05.960830 1718 raft_consensus.cc:2258] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:05.961426 1718 ts_tablet_manager.cc:1875] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:05.963284 1718 ts_tablet_manager.cc:1888] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:05.963366 1718 log.cc:1196] T 6aff1d7fd2d2476095ef0a0529110b49 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/6aff1d7fd2d2476095ef0a0529110b49
I20240501 19:57:05.963871 26910 catalog_manager.cc:4830] TS 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155): tablet 6aff1d7fd2d2476095ef0a0529110b49 (table rebalance_test_table_8 [id=88d5c82f1284436383b133473ea1e7fc]) successfully deleted
I20240501 19:57:06.052464 27066 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1"
dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53032
I20240501 19:57:06.052582 27066 raft_consensus.cc:595] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 LEADER]: Received request to transfer leadership
I20240501 19:57:06.066735 27066 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1"
dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53044
I20240501 19:57:06.066859 27066 raft_consensus.cc:595] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 LEADER]: Received request to transfer leadership
I20240501 19:57:06.069411 1517 raft_consensus.cc:979] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e: : Instructing follower e5057264da53436ea17bad01c66a7fbf to start an election
I20240501 19:57:06.069506 1517 raft_consensus.cc:1067] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 LEADER]: Signalling peer e5057264da53436ea17bad01c66a7fbf to start an election
I20240501 19:57:06.069926 27458 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1"
dest_uuid: "e5057264da53436ea17bad01c66a7fbf"
from {username='slave'} at 127.20.210.1:37061
I20240501 19:57:06.070055 27458 raft_consensus.cc:482] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 19:57:06.070112 27458 raft_consensus.cc:3037] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 2 FOLLOWER]: Advancing to term 3
I20240501 19:57:06.070936 27458 raft_consensus.cc:504] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:06.071293 27458 leader_election.cc:288] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 3 election: Requested vote from peers 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375), ecdc0d100f5f4b95b9dda10c2f97cf20 (127.20.210.7:33695), 045b41d2fb5545fd9975510efab305aa (127.20.210.9:42833)
I20240501 19:57:06.071596 29008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20"
I20240501 19:57:06.071722 29008 raft_consensus.cc:3037] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 2 FOLLOWER]: Advancing to term 3
I20240501 19:57:06.071710 30036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "045b41d2fb5545fd9975510efab305aa"
I20240501 19:57:06.071816 30036 raft_consensus.cc:3037] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Advancing to term 3
I20240501 19:57:06.072481 29008 raft_consensus.cc:2454] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 3.
I20240501 19:57:06.072546 30036 raft_consensus.cc:2454] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 3.
I20240501 19:57:06.072911 27066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "692732b291b441029826cf20c9a0ecb1" candidate_uuid: "e5057264da53436ea17bad01c66a7fbf" candidate_term: 3 candidate_status { last_received { term: 2 index: 9 } } ignore_live_leader: true dest_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e"
I20240501 19:57:06.072932 27395 leader_election.cc:302] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 045b41d2fb5545fd9975510efab305aa, e5057264da53436ea17bad01c66a7fbf, ecdc0d100f5f4b95b9dda10c2f97cf20; no voters:
I20240501 19:57:06.073022 27066 raft_consensus.cc:3032] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 LEADER]: Stepping down as leader of term 2
I20240501 19:57:06.073072 27066 raft_consensus.cc:729] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 16cfc5a5082742c5b09d35e5c4f7f10e, State: Running, Role: LEADER
I20240501 19:57:06.073107 1600 raft_consensus.cc:2781] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 3 FOLLOWER]: Leader election won for term 3
I20240501 19:57:06.073216 1600 raft_consensus.cc:686] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 3 LEADER]: Becoming Leader. State: Replica: e5057264da53436ea17bad01c66a7fbf, State: Running, Role: LEADER
I20240501 19:57:06.073221 27066 consensus_queue.cc:260] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:06.073347 1600 consensus_queue.cc:237] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 9, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:06.073410 27066 raft_consensus.cc:3037] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 2 FOLLOWER]: Advancing to term 3
I20240501 19:57:06.074306 27066 raft_consensus.cc:2454] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5057264da53436ea17bad01c66a7fbf in term 3.
I20240501 19:57:06.075829 26924 catalog_manager.cc:5484] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf reported cstate change: term changed from 2 to 3, leader changed from 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) to e5057264da53436ea17bad01c66a7fbf (127.20.210.4). New cstate: current_term: 3 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:06.315194 1767 raft_consensus.cc:1050] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf: attempting to promote NON_VOTER 045b41d2fb5545fd9975510efab305aa to VOTER
I20240501 19:57:06.315672 1767 consensus_queue.cc:237] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:06.317118 27326 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:06.317198 30036 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa [term 1 LEARNER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:06.317448 29140 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:06.317595 1769 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:06.317795 1600 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:06.318465 1600 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:06.320081 1769 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:06.320479 27326 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:06.322438 29140 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:06.322679 26924 catalog_manager.cc:5484] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:06.324476 30036 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 045b41d2fb5545fd9975510efab305aa (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3f70fdad7b438a841f74fc638d17f5" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 46155 } attrs { replace: true } } peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:06.327836 27458 consensus_queue.cc:237] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:06.328850 30036 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:06.328845 29140 raft_consensus.cc:1261] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:06.329182 1769 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:06.329339 1755 consensus_queue.cc:1035] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:06.330791 1600 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P e5057264da53436ea17bad01c66a7fbf [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:06.331223 29140 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:06.331413 30036 raft_consensus.cc:2932] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:06.333709 26916 catalog_manager.cc:5484] T 6ee4db743833421e8120edbec4ba0836 P 045b41d2fb5545fd9975510efab305aa reported cstate change: config changed from index 8 to 9, VOTER 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:06.334409 26911 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6ee4db743833421e8120edbec4ba0836 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:06.335708 27307 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ee4db743833421e8120edbec4ba0836 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3f70fdad7b438a841f74fc638d17f5 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:34348
I20240501 19:57:06.335888 1718 tablet_replica.cc:330] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5: stopping tablet replica
I20240501 19:57:06.335988 1718 raft_consensus.cc:2229] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:06.336122 1718 raft_consensus.cc:2258] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:06.336793 1718 ts_tablet_manager.cc:1875] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:06.338824 1718 ts_tablet_manager.cc:1888] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240501 19:57:06.338891 1718 log.cc:1196] T 6ee4db743833421e8120edbec4ba0836 P 3c3f70fdad7b438a841f74fc638d17f5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/6ee4db743833421e8120edbec4ba0836
I20240501 19:57:06.339320 26910 catalog_manager.cc:4830] TS 3c3f70fdad7b438a841f74fc638d17f5 (127.20.210.3:46155): tablet 6ee4db743833421e8120edbec4ba0836 (table rebalance_test_table_3 [id=934ed3cd10224c0892399e32f9a17cf4]) successfully deleted
I20240501 19:57:06.588914 29008 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 3 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20240501 19:57:06.589313 1755 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:06.643193 27066 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 3 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20240501 19:57:06.643599 1755 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "16cfc5a5082742c5b09d35e5c4f7f10e" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 33375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:06.646035 30036 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa [term 3 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20240501 19:57:06.646536 1769 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:07.188722 27458 consensus_queue.cc:237] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 3.10, Last appended by leader: 9, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } }
I20240501 19:57:07.189921 30036 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa [term 3 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 3 index: 10. Preceding OpId from leader: term: 3 index: 11. (index mismatch)
I20240501 19:57:07.190037 29007 raft_consensus.cc:1261] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 3 FOLLOWER]: Refusing update from remote peer e5057264da53436ea17bad01c66a7fbf: Log matching property violated. Preceding OpId in replica: term: 3 index: 10. Preceding OpId from leader: term: 3 index: 11. (index mismatch)
I20240501 19:57:07.190310 1810 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:07.190475 1600 consensus_queue.cc:1035] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:07.191959 1769 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf [term 3 LEADER]: Committing config change with OpId 3.11: config changed from index 9 to 11, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:07.192358 30036 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P 045b41d2fb5545fd9975510efab305aa [term 3 FOLLOWER]: Committing config change with OpId 3.11: config changed from index 9 to 11, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:07.193317 26911 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 692732b291b441029826cf20c9a0ecb1 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:07.193672 29007 raft_consensus.cc:2932] T 692732b291b441029826cf20c9a0ecb1 P ecdc0d100f5f4b95b9dda10c2f97cf20 [term 3 FOLLOWER]: Committing config change with OpId 3.11: config changed from index 9 to 11, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } }
I20240501 19:57:07.195902 26921 catalog_manager.cc:5484] T 692732b291b441029826cf20c9a0ecb1 P e5057264da53436ea17bad01c66a7fbf reported cstate change: config changed from index 9 to 11, VOTER 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1) evicted. New cstate: current_term: 3 leader_uuid: "e5057264da53436ea17bad01c66a7fbf" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "e5057264da53436ea17bad01c66a7fbf" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 37115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecdc0d100f5f4b95b9dda10c2f97cf20" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 33695 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:07.198819 27045 tablet_service.cc:1514] Processing DeleteTablet for tablet 692732b291b441029826cf20c9a0ecb1 with delete_type TABLET_DATA_TOMBSTONED (TS 16cfc5a5082742c5b09d35e5c4f7f10e not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:49644
I20240501 19:57:07.199157 1815 tablet_replica.cc:330] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e: stopping tablet replica
I20240501 19:57:07.199291 1815 raft_consensus.cc:2229] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 3 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:07.199398 1815 raft_consensus.cc:2258] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e [term 3 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:07.200032 1815 ts_tablet_manager.cc:1875] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:07.201834 1815 ts_tablet_manager.cc:1888] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.10
I20240501 19:57:07.201896 1815 log.cc:1196] T 692732b291b441029826cf20c9a0ecb1 P 16cfc5a5082742c5b09d35e5c4f7f10e: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/692732b291b441029826cf20c9a0ecb1
I20240501 19:57:07.202378 26909 catalog_manager.cc:4830] TS 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375): tablet 692732b291b441029826cf20c9a0ecb1 (table rebalance_test_table_0 [id=891277205119418895f5fad0ed146e21]) successfully deleted
I20240501 19:57:07.388603 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:07.389508 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:07.389972 30019 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:07.390383 29120 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 19:57:07.396083 27307 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 19:57:07.403681 27437 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:07.411937 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:07.412374 28988 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:07.436311 27175 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:07.468894 29139 consensus_queue.cc:237] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:57:07.470136 30036 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 19:57:07.470170 27196 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
W20240501 19:57:07.470371 29074 consensus_peers.cc:485] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 -> Peer 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529): Couldn't send request to peer 7ccb4fe66ab942d399551b10f5ed32ea. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5a9674523a2148b1a6c1a5d22a57aac5. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:07.470574 1875 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:07.470782 1874 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:07.472292 1875 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 2 LEADER]: Committing config change with OpId 2.11: config changed from index 10 to 11, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:57:07.472672 30036 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 10 to 11, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:57:07.474812 26918 catalog_manager.cc:5484] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa reported cstate change: config changed from index 10 to 11, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New cstate: current_term: 2 leader_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:57:07.475662 27196 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 10 to 11, NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } } }
I20240501 19:57:08.034626 1894 ts_tablet_manager.cc:902] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea: Initiating tablet copy from peer 474eb4491dcb47f4a2a2f6e149bc39c1 (127.20.210.8:44887)
I20240501 19:57:08.035071 1894 tablet_copy_client.cc:308] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.8:44887
I20240501 19:57:08.038103 29160 tablet_copy_service.cc:140] P 474eb4491dcb47f4a2a2f6e149bc39c1: Received BeginTabletCopySession request for tablet 5a9674523a2148b1a6c1a5d22a57aac5 from peer 7ccb4fe66ab942d399551b10f5ed32ea ({username='slave'} at 127.20.210.6:35205)
I20240501 19:57:08.038223 29160 tablet_copy_service.cc:161] P 474eb4491dcb47f4a2a2f6e149bc39c1: Beginning new tablet copy session on tablet 5a9674523a2148b1a6c1a5d22a57aac5 from peer 7ccb4fe66ab942d399551b10f5ed32ea at {username='slave'} at 127.20.210.6:35205: session id = 7ccb4fe66ab942d399551b10f5ed32ea-5a9674523a2148b1a6c1a5d22a57aac5
I20240501 19:57:08.039296 29160 tablet_copy_source_session.cc:205] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:08.040287 1894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a9674523a2148b1a6c1a5d22a57aac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:08.042053 1894 tablet_copy_client.cc:783] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:08.042208 1894 tablet_copy_client.cc:647] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:08.043390 1898 tablet_copy_client.cc:1002] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Injecting latency into file download: 1500
I20240501 19:57:09.543648 1894 tablet_copy_client.cc:523] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:09.544865 1894 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap starting.
I20240501 19:57:09.562379 1894 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:09.562626 1894 tablet_bootstrap.cc:492] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea: Bootstrap complete.
I20240501 19:57:09.562790 1894 ts_tablet_manager.cc:1367] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.005s
I20240501 19:57:09.563148 1894 raft_consensus.cc:348] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:57:09.563251 1894 raft_consensus.cc:729] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 7ccb4fe66ab942d399551b10f5ed32ea, State: Initialized, Role: LEARNER
I20240501 19:57:09.563355 1894 consensus_queue.cc:260] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: true } }
I20240501 19:57:09.563661 1894 ts_tablet_manager.cc:1398] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:09.564253 29160 tablet_copy_service.cc:342] P 474eb4491dcb47f4a2a2f6e149bc39c1: Request end of tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-5a9674523a2148b1a6c1a5d22a57aac5 received from {username='slave'} at 127.20.210.6:35205
I20240501 19:57:09.564359 29160 tablet_copy_service.cc:433] P 474eb4491dcb47f4a2a2f6e149bc39c1: ending tablet copy session 7ccb4fe66ab942d399551b10f5ed32ea-5a9674523a2148b1a6c1a5d22a57aac5 on tablet 5a9674523a2148b1a6c1a5d22a57aac5 with peer 7ccb4fe66ab942d399551b10f5ed32ea
I20240501 19:57:09.583395 28876 raft_consensus.cc:1203] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea [term 2 LEARNER]: Deduplicated request from leader. Original: 2.10->[2.11-2.11] Dedup: 2.11->[]
I20240501 19:57:09.983892 1920 raft_consensus.cc:1050] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1: attempting to promote NON_VOTER 7ccb4fe66ab942d399551b10f5ed32ea to VOTER
I20240501 19:57:09.984371 1920 consensus_queue.cc:237] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:57:09.985603 28876 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea [term 2 LEARNER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20240501 19:57:09.985842 27196 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20240501 19:57:09.986068 1963 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:09.986143 30039 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20240501 19:57:09.986485 1960 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:09.986647 1963 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:09.988283 1960 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 2 LEADER]: Committing config change with OpId 2.12: config changed from index 11 to 12, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:09.988778 27196 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Committing config change with OpId 2.12: config changed from index 11 to 12, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:09.988942 30039 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Committing config change with OpId 2.12: config changed from index 11 to 12, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:09.990988 26918 catalog_manager.cc:5484] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 reported cstate change: config changed from index 11 to 12, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:09.991047 28876 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea [term 2 FOLLOWER]: Committing config change with OpId 2.12: config changed from index 11 to 12, 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ef2d33d636574e8fbb9d3b90a15035d6" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 43113 } attrs { replace: true } } peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:09.995271 29139 consensus_queue.cc:237] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 2.12, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } }
I20240501 19:57:09.996201 28876 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20240501 19:57:09.996285 30039 raft_consensus.cc:1261] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Refusing update from remote peer 474eb4491dcb47f4a2a2f6e149bc39c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20240501 19:57:09.996528 1962 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:09.996696 1960 consensus_queue.cc:1035] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:09.998092 1920 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 474eb4491dcb47f4a2a2f6e149bc39c1 [term 2 LEADER]: Committing config change with OpId 2.13: config changed from index 12 to 13, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:09.998512 28876 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 7ccb4fe66ab942d399551b10f5ed32ea [term 2 FOLLOWER]: Committing config change with OpId 2.13: config changed from index 12 to 13, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:09.998586 30039 raft_consensus.cc:2932] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa [term 2 FOLLOWER]: Committing config change with OpId 2.13: config changed from index 12 to 13, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:09.999378 26910 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5a9674523a2148b1a6c1a5d22a57aac5 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:10.000592 26921 catalog_manager.cc:5484] T 5a9674523a2148b1a6c1a5d22a57aac5 P 045b41d2fb5545fd9975510efab305aa reported cstate change: config changed from index 12 to 13, VOTER ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2) evicted. New cstate: current_term: 2 leader_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "474eb4491dcb47f4a2a2f6e149bc39c1" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 44887 } attrs { promote: false } } peers { permanent_uuid: "045b41d2fb5545fd9975510efab305aa" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 42833 } attrs { promote: false } } peers { permanent_uuid: "7ccb4fe66ab942d399551b10f5ed32ea" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 38529 } attrs { promote: false } } }
I20240501 19:57:10.002524 27175 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a9674523a2148b1a6c1a5d22a57aac5 with delete_type TABLET_DATA_TOMBSTONED (TS ef2d33d636574e8fbb9d3b90a15035d6 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:43438
I20240501 19:57:10.002823 1972 tablet_replica.cc:330] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6: stopping tablet replica
I20240501 19:57:10.002966 1972 raft_consensus.cc:2229] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:10.003159 1972 raft_consensus.cc:2258] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:10.003639 1972 ts_tablet_manager.cc:1875] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:10.005339 1972 ts_tablet_manager.cc:1888] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.12
I20240501 19:57:10.005399 1972 log.cc:1196] T 5a9674523a2148b1a6c1a5d22a57aac5 P ef2d33d636574e8fbb9d3b90a15035d6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/5a9674523a2148b1a6c1a5d22a57aac5
I20240501 19:57:10.005836 26908 catalog_manager.cc:4830] TS ef2d33d636574e8fbb9d3b90a15035d6 (127.20.210.2:43113): tablet 5a9674523a2148b1a6c1a5d22a57aac5 (table rebalance_test_table_5 [id=15f2762ca1b64f9ea955e5c695654799]) successfully deleted
I20240501 19:57:10.220769 30019 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.222548 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:10.223392 29120 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 19:57:10.232414 28988 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:10.232481 27307 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 19:57:10.242465 27175 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:10.249846 27437 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:10.256641 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.269053 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.352618 27307 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 19:57:10.358821 29120 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 19:57:10.359969 30019 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.366432 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.368885 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:10.375501 27437 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:10.389354 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.396570 28988 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:10.399861 27175 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:10.626749 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:10.631443 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.632067 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.636473 27175 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:10.650426 27307 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 19:57:10.651757 29120 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 19:57:10.653297 27437 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:10.654978 28988 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:10.666486 30019 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.771044 30019 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.777953 29120 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 19:57:10.779235 27307 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 19:57:10.780930 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:10.781060 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.788635 27175 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:10.794651 28988 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:10.798903 27437 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:10.820078 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.945385 30019 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.949426 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:10.956382 29120 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 19:57:10.959278 28988 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:10.966475 27175 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:10.967092 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:10.968164 27307 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240501 19:57:10.975837 27437 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:10.989995 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:11.091253 30019 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:11.094280 27045 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:11.095775 28726 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:11.101275 28857 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:11.102015 29120 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240501 19:57:11.103473 27437 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:11.118988 28988 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:11.122185 27175 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:11.136682 27307 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
8496c8b9092949be83f5731019951f7e | 127.20.210.62:43635 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.20.210.20:44805 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
045b41d2fb5545fd9975510efab305aa | 127.20.210.9:42833 | HEALTHY | <none> | 0 | 0
16cfc5a5082742c5b09d35e5c4f7f10e | 127.20.210.1:33375 | HEALTHY | <none> | 6 | 0
3c3f70fdad7b438a841f74fc638d17f5 | 127.20.210.3:46155 | HEALTHY | <none> | 8 | 0
474eb4491dcb47f4a2a2f6e149bc39c1 | 127.20.210.8:44887 | HEALTHY | <none> | 2 | 0
724bae28280242c6982c4f8e315b0d0b | 127.20.210.5:42343 | HEALTHY | <none> | 0 | 0
7ccb4fe66ab942d399551b10f5ed32ea | 127.20.210.6:38529 | HEALTHY | <none> | 0 | 0
e5057264da53436ea17bad01c66a7fbf | 127.20.210.4:37115 | HEALTHY | <none> | 6 | 0
ecdc0d100f5f4b95b9dda10c2f97cf20 | 127.20.210.7:33695 | HEALTHY | <none> | 3 | 0
ef2d33d636574e8fbb9d3b90a15035d6 | 127.20.210.2:43113 | HEALTHY | <none> | 5 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 9
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
---------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 9 server(s) checked
local_ip_for_outbound_sockets | 127.20.210.1 | experimental | 127.20.210.1:33375
local_ip_for_outbound_sockets | 127.20.210.2 | experimental | 127.20.210.2:43113
local_ip_for_outbound_sockets | 127.20.210.3 | experimental | 127.20.210.3:46155
local_ip_for_outbound_sockets | 127.20.210.4 | experimental | 127.20.210.4:37115
local_ip_for_outbound_sockets | 127.20.210.5 | experimental | 127.20.210.5:42343
local_ip_for_outbound_sockets | 127.20.210.6 | experimental | 127.20.210.6:38529
local_ip_for_outbound_sockets | 127.20.210.7 | experimental | 127.20.210.7:33695
local_ip_for_outbound_sockets | 127.20.210.8 | experimental | 127.20.210.8:44887
local_ip_for_outbound_sockets | 127.20.210.9 | experimental | 127.20.210.9:42833
never_fsync | true | unsafe,advanced | all 9 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 9 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 9 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 9 server(s) checked
server_dump_info_format | pb | hidden | all 9 server(s) checked
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.20.210.1:33375
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.20.210.2:43113
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.20.210.3:46155
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.20.210.4:37115
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.20.210.5:42343
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.20.210.6:38529
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.20.210.7:33695
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.20.210.8:44887
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/info.pb | hidden | 127.20.210.9:42833
tablet_copy_download_file_inject_latency_ms | 1500 | hidden | all 9 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.20.210.20:44805 | all 9 server(s) checked
time_source | builtin | all 9 server(s) checked
Version Summary
Version | Servers
-----------------+--------------------------
1.18.0-SNAPSHOT | all 10 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
rebalance_test_table_0 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_5 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_6 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_7 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_8 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_9 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 10
First Quartile | 10
Median | 10
Third Quartile | 10
Maximum | 10
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 9
Tables | 10
Tablets | 30
Replicas | 90
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240501 19:57:11.163823 21320 log_verifier.cc:126] Checking tablet 0723850372f8416d9b8ac19f119939bc
I20240501 19:57:11.191761 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 0723850372f8416d9b8ac19f119939bc
I20240501 19:57:11.191833 21320 log_verifier.cc:126] Checking tablet 0aaa59df03844aadafb8d57797685dc2
I20240501 19:57:11.215507 21320 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 0aaa59df03844aadafb8d57797685dc2
I20240501 19:57:11.215564 21320 log_verifier.cc:126] Checking tablet 0f417f4001bc48158187ed262f7737ab
I20240501 19:57:11.238502 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 0f417f4001bc48158187ed262f7737ab
I20240501 19:57:11.238557 21320 log_verifier.cc:126] Checking tablet 199f9529870242f599358995c19250b2
I20240501 19:57:11.261443 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 199f9529870242f599358995c19250b2
I20240501 19:57:11.261497 21320 log_verifier.cc:126] Checking tablet 360f349692784ae69f01ddb9afe73da4
I20240501 19:57:11.284459 21320 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 360f349692784ae69f01ddb9afe73da4
I20240501 19:57:11.284518 21320 log_verifier.cc:126] Checking tablet 37288cafed764ba18fcc07b6e3c7a722
I20240501 19:57:11.308914 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 37288cafed764ba18fcc07b6e3c7a722
I20240501 19:57:11.308977 21320 log_verifier.cc:126] Checking tablet 3c80c1dcbcd3439ba75066259557e716
I20240501 19:57:11.331837 21320 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 3c80c1dcbcd3439ba75066259557e716
I20240501 19:57:11.331892 21320 log_verifier.cc:126] Checking tablet 3ca822a1462544c3be45371d6e088447
I20240501 19:57:11.356096 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 3ca822a1462544c3be45371d6e088447
I20240501 19:57:11.356158 21320 log_verifier.cc:126] Checking tablet 3e05a10bfc6540c19faa154896b8976a
I20240501 19:57:11.380311 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 3e05a10bfc6540c19faa154896b8976a
I20240501 19:57:11.380367 21320 log_verifier.cc:126] Checking tablet 55d39ed3702046b08b1ac3c76c272c01
I20240501 19:57:11.405915 21320 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 55d39ed3702046b08b1ac3c76c272c01
I20240501 19:57:11.405974 21320 log_verifier.cc:126] Checking tablet 5a9674523a2148b1a6c1a5d22a57aac5
I20240501 19:57:11.429694 21320 log_verifier.cc:177] Verified matching terms for 13 ops in tablet 5a9674523a2148b1a6c1a5d22a57aac5
I20240501 19:57:11.429762 21320 log_verifier.cc:126] Checking tablet 5ecba34e358c41d9b812113c955b6b29
I20240501 19:57:11.452879 21320 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 5ecba34e358c41d9b812113c955b6b29
I20240501 19:57:11.452936 21320 log_verifier.cc:126] Checking tablet 6211064bb3ce41f8b19729d8d0de7fe8
I20240501 19:57:11.475798 21320 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 6211064bb3ce41f8b19729d8d0de7fe8
I20240501 19:57:11.475857 21320 log_verifier.cc:126] Checking tablet 692732b291b441029826cf20c9a0ecb1
I20240501 19:57:11.501717 21320 log_verifier.cc:177] Verified matching terms for 11 ops in tablet 692732b291b441029826cf20c9a0ecb1
I20240501 19:57:11.501780 21320 log_verifier.cc:126] Checking tablet 6aff1d7fd2d2476095ef0a0529110b49
I20240501 19:57:11.525307 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 6aff1d7fd2d2476095ef0a0529110b49
I20240501 19:57:11.525364 21320 log_verifier.cc:126] Checking tablet 6c498eacc5e947bfb01dfb90c7039df0
I20240501 19:57:11.548339 21320 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 6c498eacc5e947bfb01dfb90c7039df0
I20240501 19:57:11.548400 21320 log_verifier.cc:126] Checking tablet 6ee4db743833421e8120edbec4ba0836
I20240501 19:57:11.571269 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 6ee4db743833421e8120edbec4ba0836
I20240501 19:57:11.571346 21320 log_verifier.cc:126] Checking tablet 75cc96baf4654318a09dcc34eb32819c
I20240501 19:57:11.594563 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 75cc96baf4654318a09dcc34eb32819c
I20240501 19:57:11.594633 21320 log_verifier.cc:126] Checking tablet 82f387c0d03a473fb548b447dc867cc7
I20240501 19:57:11.616782 21320 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 82f387c0d03a473fb548b447dc867cc7
I20240501 19:57:11.616851 21320 log_verifier.cc:126] Checking tablet 896072a285b649dbb31f9f02666ace25
I20240501 19:57:11.640447 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 896072a285b649dbb31f9f02666ace25
I20240501 19:57:11.640503 21320 log_verifier.cc:126] Checking tablet 8c157f7f72dc4211b256eee0015eed23
I20240501 19:57:11.663342 21320 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 8c157f7f72dc4211b256eee0015eed23
I20240501 19:57:11.663393 21320 log_verifier.cc:126] Checking tablet 9102f7c8350843c2bd7009ea190860d0
I20240501 19:57:11.686836 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 9102f7c8350843c2bd7009ea190860d0
I20240501 19:57:11.686892 21320 log_verifier.cc:126] Checking tablet a74a8248e5584fb483728f7676502ad5
I20240501 19:57:11.708909 21320 log_verifier.cc:177] Verified matching terms for 6 ops in tablet a74a8248e5584fb483728f7676502ad5
I20240501 19:57:11.708962 21320 log_verifier.cc:126] Checking tablet ae366b11dc9f492b9467d68947cd3dc1
I20240501 19:57:11.731905 21320 log_verifier.cc:177] Verified matching terms for 6 ops in tablet ae366b11dc9f492b9467d68947cd3dc1
I20240501 19:57:11.731972 21320 log_verifier.cc:126] Checking tablet b2c944ca787b475294f9c41f6052a7ea
I20240501 19:57:11.755332 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet b2c944ca787b475294f9c41f6052a7ea
I20240501 19:57:11.755393 21320 log_verifier.cc:126] Checking tablet b3e84419a13348e4a0e7bacfe94cf33e
I20240501 19:57:11.779199 21320 log_verifier.cc:177] Verified matching terms for 7 ops in tablet b3e84419a13348e4a0e7bacfe94cf33e
I20240501 19:57:11.779268 21320 log_verifier.cc:126] Checking tablet c9f961f0554c45a1a3f31d5a8b31a46d
I20240501 19:57:11.804055 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet c9f961f0554c45a1a3f31d5a8b31a46d
I20240501 19:57:11.804121 21320 log_verifier.cc:126] Checking tablet e379c346ae524bd28f6e550051c0bf7e
I20240501 19:57:11.829627 21320 log_verifier.cc:177] Verified matching terms for 6 ops in tablet e379c346ae524bd28f6e550051c0bf7e
I20240501 19:57:11.829684 21320 log_verifier.cc:126] Checking tablet e3b87a633efe4400931f1e0a566c8f14
I20240501 19:57:11.854568 21320 log_verifier.cc:177] Verified matching terms for 6 ops in tablet e3b87a633efe4400931f1e0a566c8f14
I20240501 19:57:11.854622 21320 log_verifier.cc:126] Checking tablet f149fd8790a0412e81ac49d0ce359ede
I20240501 19:57:11.879707 21320 log_verifier.cc:177] Verified matching terms for 9 ops in tablet f149fd8790a0412e81ac49d0ce359ede
I20240501 19:57:11.880085 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 26984
I20240501 19:57:11.887300 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 27115
I20240501 19:57:11.894073 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 27246
I20240501 19:57:11.900931 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 27377
I20240501 19:57:11.907884 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 28664
I20240501 19:57:11.913281 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 28795
W20240501 19:57:11.916508 28942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.210.1:33375: connect: Connection refused (error 111)
W20240501 19:57:11.917263 28942 consensus_peers.cc:485] T 6211064bb3ce41f8b19729d8d0de7fe8 P ecdc0d100f5f4b95b9dda10c2f97cf20 -> Peer 16cfc5a5082742c5b09d35e5c4f7f10e (127.20.210.1:33375): Couldn't send request to peer 16cfc5a5082742c5b09d35e5c4f7f10e. Status: Network error: Client connection negotiation failed: client connection to 127.20.210.1:33375: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:11.918794 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 28926
W20240501 19:57:11.921453 29074 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.210.6:38529: connect: Connection refused (error 111)
W20240501 19:57:11.922277 29074 consensus_peers.cc:485] T 0aaa59df03844aadafb8d57797685dc2 P 474eb4491dcb47f4a2a2f6e149bc39c1 -> Peer 7ccb4fe66ab942d399551b10f5ed32ea (127.20.210.6:38529): Couldn't send request to peer 7ccb4fe66ab942d399551b10f5ed32ea. Status: Network error: Client connection negotiation failed: client connection to 127.20.210.6:38529: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:11.924641 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 29057
I20240501 19:57:11.930263 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 29842
I20240501 19:57:11.935344 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 26893
2024-05-01T19:57:11Z chronyd exiting
[ OK ] TserverAddedDuringRebalancingTest.TserverStarts/1 (32438 ms)
[----------] 1 test from TserverAddedDuringRebalancingTest (32438 ms total)
[----------] 1 test from RebalancerAndSingleReplicaTablets
[ RUN ] RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove/5
I20240501 19:57:12.034229 21320 test_util.cc:268] Using random seed: -462201935
I20240501 19:57:12.034924 21320 ts_itest-base.cc:112] Starting cluster with:
I20240501 19:57:12.034957 21320 ts_itest-base.cc:113] --------------
I20240501 19:57:12.034967 21320 ts_itest-base.cc:114] 4 tablet servers
I20240501 19:57:12.034977 21320 ts_itest-base.cc:115] 1 replicas per TS
I20240501 19:57:12.034986 21320 ts_itest-base.cc:116] --------------
2024-05-01T19:57:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T19:57:12Z Disabled control of system clock
I20240501 19:57:12.042951 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:44939
--webserver_interface=127.20.210.62
--webserver_port=0
--builtin_ntp_servers=127.20.210.20:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.210.62:44939
--raft_prepare_replacement_before_eviction=true
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false with env {}
W20240501 19:57:12.137701 2251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:12.137873 2251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:12.137928 2251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:12.140668 2251 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:12.140720 2251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 19:57:12.140741 2251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:12.140763 2251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 19:57:12.140779 2251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 19:57:12.143970 2251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.20.210.62:44939
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.20.210.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.2251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:12.144635 2251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:12.145501 2251 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240501 19:57:12.154273 2251 server_base.cc:1034] running on GCE node
W20240501 19:57:12.155002 2256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:12.155508 2259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:12.156224 2257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:12.156644 2251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:12.157430 2251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:12.158576 2251 hybrid_clock.cc:648] HybridClock initialized: now 1714593432158551 us; error 39 us; skew 500 ppm
I20240501 19:57:12.160274 2251 webserver.cc:460] Webserver started at http://127.20.210.62:42245/ using document root <none> and password file <none>
I20240501 19:57:12.160746 2251 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:12.160813 2251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:12.160988 2251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:12.162443 2251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "1f4f92ff7412455581cdfc3f66658434"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.162859 2251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "1f4f92ff7412455581cdfc3f66658434"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.165688 2251 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 19:57:12.167575 2265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:12.168445 2251 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:12.168556 2251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
uuid: "1f4f92ff7412455581cdfc3f66658434"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.168628 2251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:12.207181 2251 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:12.207690 2251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:12.207856 2251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:12.213492 2251 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.62:44939
I20240501 19:57:12.213507 2318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.62:44939 every 8 connection(s)
I20240501 19:57:12.214267 2251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 19:57:12.216324 2319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:12.217428 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 2251
I20240501 19:57:12.217571 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 19:57:12.221272 2319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434: Bootstrap starting.
I20240501 19:57:12.223306 2319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:12.224025 2319 log.cc:824] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:12.225523 2319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434: No bootstrap required, opened a new log
I20240501 19:57:12.227540 2319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f4f92ff7412455581cdfc3f66658434" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 44939 } }
I20240501 19:57:12.227710 2319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:12.227794 2319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f4f92ff7412455581cdfc3f66658434, State: Initialized, Role: FOLLOWER
I20240501 19:57:12.228149 2319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f4f92ff7412455581cdfc3f66658434" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 44939 } }
I20240501 19:57:12.228262 2319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:12.228325 2319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:12.228430 2319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:12.229077 2319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f4f92ff7412455581cdfc3f66658434" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 44939 } }
I20240501 19:57:12.229358 2319 leader_election.cc:302] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f4f92ff7412455581cdfc3f66658434; no voters:
I20240501 19:57:12.229579 2319 leader_election.cc:288] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:12.229854 2324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:12.230027 2324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [term 1 LEADER]: Becoming Leader. State: Replica: 1f4f92ff7412455581cdfc3f66658434, State: Running, Role: LEADER
I20240501 19:57:12.230289 2324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f4f92ff7412455581cdfc3f66658434" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 44939 } }
I20240501 19:57:12.230458 2319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 19:57:12.231580 2325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f4f92ff7412455581cdfc3f66658434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f4f92ff7412455581cdfc3f66658434" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 44939 } } }
I20240501 19:57:12.231669 2325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [sys.catalog]: This master's current role is: LEADER
I20240501 19:57:12.231616 2326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f4f92ff7412455581cdfc3f66658434. Latest consensus state: current_term: 1 leader_uuid: "1f4f92ff7412455581cdfc3f66658434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f4f92ff7412455581cdfc3f66658434" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 44939 } } }
I20240501 19:57:12.231794 2326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434 [sys.catalog]: This master's current role is: LEADER
W20240501 19:57:12.234200 2339 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240501 19:57:12.234287 2339 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240501 19:57:12.234349 2334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 19:57:12.235123 2334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 19:57:12.239041 2334 catalog_manager.cc:1347] Generated new cluster ID: ddc2b32c478241d886e6106be54460d8
I20240501 19:57:12.239097 2334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 19:57:12.246866 2334 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 19:57:12.247673 2334 catalog_manager.cc:1504] Loading token signing keys...
I20240501 19:57:12.252604 2334 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 1f4f92ff7412455581cdfc3f66658434: Generated new TSK 0
I20240501 19:57:12.253068 2334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 19:57:12.255748 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.1:0
--local_ip_for_outbound_sockets=127.20.210.1
--webserver_interface=127.20.210.1
--webserver_port=0
--tserver_master_addrs=127.20.210.62:44939
--builtin_ntp_servers=127.20.210.20:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 19:57:12.349300 2343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:12.349474 2343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:12.349541 2343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:12.352303 2343 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:12.352365 2343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:12.352439 2343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.1
I20240501 19:57:12.355494 2343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.20.210.1
--webserver_port=0
--tserver_master_addrs=127.20.210.62:44939
--never_fsync=true
--heap_profile_path=/tmp/kudu.2343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.1
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:12.356127 2343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:12.357023 2343 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:12.364585 2351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:12.365422 2348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:12.366076 2349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:12.366747 2343 server_base.cc:1034] running on GCE node
I20240501 19:57:12.367194 2343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:12.367846 2343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:12.369022 2343 hybrid_clock.cc:648] HybridClock initialized: now 1714593432369006 us; error 31 us; skew 500 ppm
I20240501 19:57:12.370828 2343 webserver.cc:460] Webserver started at http://127.20.210.1:37609/ using document root <none> and password file <none>
I20240501 19:57:12.371455 2343 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:12.371516 2343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:12.371693 2343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:12.373526 2343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "4b1f3070b19c49cb8cc3cce7e3d399db"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.374009 2343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "4b1f3070b19c49cb8cc3cce7e3d399db"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.377756 2343 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 19:57:12.379937 2357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:12.380789 2343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:12.380872 2343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "4b1f3070b19c49cb8cc3cce7e3d399db"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.380932 2343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:12.418620 2343 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:12.419092 2343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:12.419209 2343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:12.419610 2343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:12.420341 2343 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:12.420387 2343 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:12.420420 2343 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:12.420441 2343 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:12.427811 2343 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.1:34267
I20240501 19:57:12.427856 2471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.1:34267 every 8 connection(s)
I20240501 19:57:12.428611 2343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 19:57:12.431046 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 2343
I20240501 19:57:12.431175 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 19:57:12.433472 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.2:0
--local_ip_for_outbound_sockets=127.20.210.2
--webserver_interface=127.20.210.2
--webserver_port=0
--tserver_master_addrs=127.20.210.62:44939
--builtin_ntp_servers=127.20.210.20:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:12.437538 2472 heartbeater.cc:347] Connected to a master server at 127.20.210.62:44939
I20240501 19:57:12.437718 2472 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:12.438163 2472 heartbeater.cc:511] Master 127.20.210.62:44939 requested a full tablet report, sending...
I20240501 19:57:12.439644 2283 ts_manager.cc:194] Registered new tserver with Master: 4b1f3070b19c49cb8cc3cce7e3d399db (127.20.210.1:34267)
I20240501 19:57:12.441272 2283 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.1:50085
W20240501 19:57:12.532395 2475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:12.532572 2475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:12.532611 2475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:12.535451 2475 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:12.535503 2475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:12.535563 2475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.2
I20240501 19:57:12.538846 2475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.20.210.2
--webserver_port=0
--tserver_master_addrs=127.20.210.62:44939
--never_fsync=true
--heap_profile_path=/tmp/kudu.2475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.2
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:12.539512 2475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:12.540405 2475 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:12.547917 2483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:12.548712 2480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:12.549468 2481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:12.550259 2475 server_base.cc:1034] running on GCE node
I20240501 19:57:12.550612 2475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:12.551101 2475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:12.552253 2475 hybrid_clock.cc:648] HybridClock initialized: now 1714593432552237 us; error 28 us; skew 500 ppm
I20240501 19:57:12.553786 2475 webserver.cc:460] Webserver started at http://127.20.210.2:35347/ using document root <none> and password file <none>
I20240501 19:57:12.554291 2475 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:12.554349 2475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:12.554512 2475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:12.555948 2475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "1b01c2708530416c96980d5b57b68993"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.556396 2475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "1b01c2708530416c96980d5b57b68993"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.559355 2475 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 19:57:12.561161 2489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:12.562047 2475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:12.562144 2475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "1b01c2708530416c96980d5b57b68993"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.562224 2475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:12.577962 2475 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:12.578428 2475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:12.578562 2475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:12.579001 2475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:12.579675 2475 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:12.579718 2475 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:12.579752 2475 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:12.579788 2475 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:12.587148 2475 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.2:32961
I20240501 19:57:12.587173 2603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.2:32961 every 8 connection(s)
I20240501 19:57:12.587896 2475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 19:57:12.588691 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 2475
I20240501 19:57:12.588798 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 19:57:12.590807 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.3:0
--local_ip_for_outbound_sockets=127.20.210.3
--webserver_interface=127.20.210.3
--webserver_port=0
--tserver_master_addrs=127.20.210.62:44939
--builtin_ntp_servers=127.20.210.20:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:12.596153 2604 heartbeater.cc:347] Connected to a master server at 127.20.210.62:44939
I20240501 19:57:12.596329 2604 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:12.596716 2604 heartbeater.cc:511] Master 127.20.210.62:44939 requested a full tablet report, sending...
I20240501 19:57:12.597609 2283 ts_manager.cc:194] Registered new tserver with Master: 1b01c2708530416c96980d5b57b68993 (127.20.210.2:32961)
I20240501 19:57:12.598464 2283 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.2:35317
W20240501 19:57:12.684020 2607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:12.684183 2607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:12.684223 2607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:12.687052 2607 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:12.687108 2607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:12.687166 2607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.3
I20240501 19:57:12.690351 2607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.20.210.3
--webserver_port=0
--tserver_master_addrs=127.20.210.62:44939
--never_fsync=true
--heap_profile_path=/tmp/kudu.2607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.3
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:12.690955 2607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:12.691803 2607 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:12.698906 2612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:12.699671 2615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:12.700373 2613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:12.701107 2607 server_base.cc:1034] running on GCE node
I20240501 19:57:12.701395 2607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:12.701833 2607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:12.702965 2607 hybrid_clock.cc:648] HybridClock initialized: now 1714593432702945 us; error 30 us; skew 500 ppm
I20240501 19:57:12.704468 2607 webserver.cc:460] Webserver started at http://127.20.210.3:34947/ using document root <none> and password file <none>
I20240501 19:57:12.704928 2607 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:12.704986 2607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:12.705117 2607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:12.706483 2607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.706854 2607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.709609 2607 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20240501 19:57:12.711354 2621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:12.712146 2607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:12.712232 2607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.712316 2607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:12.733404 2607 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:12.733800 2607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:12.733922 2607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:12.734333 2607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:12.735054 2607 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:12.735096 2607 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:12.735134 2607 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:12.735162 2607 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:12.742532 2607 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.3:38391
I20240501 19:57:12.742568 2734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.3:38391 every 8 connection(s)
I20240501 19:57:12.743294 2607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 19:57:12.745710 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 2607
I20240501 19:57:12.745811 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 19:57:12.747560 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.4:0
--local_ip_for_outbound_sockets=127.20.210.4
--webserver_interface=127.20.210.4
--webserver_port=0
--tserver_master_addrs=127.20.210.62:44939
--builtin_ntp_servers=127.20.210.20:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:12.751006 2735 heartbeater.cc:347] Connected to a master server at 127.20.210.62:44939
I20240501 19:57:12.751210 2735 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:12.751709 2735 heartbeater.cc:511] Master 127.20.210.62:44939 requested a full tablet report, sending...
I20240501 19:57:12.752630 2283 ts_manager.cc:194] Registered new tserver with Master: 9d7f7ffb3ee548faa1ef4834c1f4ee46 (127.20.210.3:38391)
I20240501 19:57:12.753129 2283 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.3:33591
W20240501 19:57:12.839634 2738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:12.839813 2738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:12.839881 2738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:12.842485 2738 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:12.842543 2738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:12.842628 2738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.4
I20240501 19:57:12.845739 2738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.4:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.20.210.4
--webserver_port=0
--tserver_master_addrs=127.20.210.62:44939
--never_fsync=true
--heap_profile_path=/tmp/kudu.2738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.4
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:12.846410 2738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:12.847271 2738 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:12.854310 2743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:12.855054 2746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:12.855707 2744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:12.856400 2738 server_base.cc:1034] running on GCE node
I20240501 19:57:12.856701 2738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:12.857159 2738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:12.858304 2738 hybrid_clock.cc:648] HybridClock initialized: now 1714593432858291 us; error 22 us; skew 500 ppm
I20240501 19:57:12.859843 2738 webserver.cc:460] Webserver started at http://127.20.210.4:44651/ using document root <none> and password file <none>
I20240501 19:57:12.860333 2738 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:12.860402 2738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:12.860549 2738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:12.861858 2738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "3729e12454ae4d6a8048c13b825e2cfc"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.862267 2738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "3729e12454ae4d6a8048c13b825e2cfc"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.865128 2738 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:12.867017 2752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:12.867869 2738 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:12.867969 2738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "3729e12454ae4d6a8048c13b825e2cfc"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:12.868045 2738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:12.882225 2738 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:12.882716 2738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:12.882858 2738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:12.883273 2738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:12.884001 2738 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:12.884054 2738 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:12.884107 2738 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:12.884152 2738 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:12.891603 2738 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.4:44677
I20240501 19:57:12.891664 2865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.4:44677 every 8 connection(s)
I20240501 19:57:12.892405 2738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 19:57:12.892812 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 2738
I20240501 19:57:12.892942 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 19:57:12.899883 2866 heartbeater.cc:347] Connected to a master server at 127.20.210.62:44939
I20240501 19:57:12.900041 2866 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:12.900455 2866 heartbeater.cc:511] Master 127.20.210.62:44939 requested a full tablet report, sending...
I20240501 19:57:12.902074 2283 ts_manager.cc:194] Registered new tserver with Master: 3729e12454ae4d6a8048c13b825e2cfc (127.20.210.4:44677)
I20240501 19:57:12.902545 2283 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.4:51469
I20240501 19:57:12.906409 21320 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240501 19:57:12.913846 2283 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48762:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240501 19:57:12.926890 2669 tablet_service.cc:1467] Processing CreateTablet for tablet 7c4bce39c03a4c5191bc8a43612ac8e6 (DEFAULT_TABLE table=TestTable [id=e1bfc967d53d48d298b31a9303bdadb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:12.927570 2669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c4bce39c03a4c5191bc8a43612ac8e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:12.931866 2879 tablet_bootstrap.cc:492] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46: Bootstrap starting.
I20240501 19:57:12.933419 2879 tablet_bootstrap.cc:654] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:12.934059 2879 log.cc:824] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:12.935274 2879 tablet_bootstrap.cc:492] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46: No bootstrap required, opened a new log
I20240501 19:57:12.935413 2879 ts_tablet_manager.cc:1367] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 19:57:12.937651 2879 raft_consensus.cc:348] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38391 } }
I20240501 19:57:12.937844 2879 raft_consensus.cc:374] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:12.937904 2879 raft_consensus.cc:729] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7f7ffb3ee548faa1ef4834c1f4ee46, State: Initialized, Role: FOLLOWER
I20240501 19:57:12.938355 2879 consensus_queue.cc:260] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38391 } }
I20240501 19:57:12.938498 2879 raft_consensus.cc:388] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:12.938561 2879 raft_consensus.cc:482] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:12.938659 2879 raft_consensus.cc:3037] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:12.939322 2879 raft_consensus.cc:504] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38391 } }
I20240501 19:57:12.939580 2879 leader_election.cc:302] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d7f7ffb3ee548faa1ef4834c1f4ee46; no voters:
I20240501 19:57:12.939801 2879 leader_election.cc:288] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:12.939875 2881 raft_consensus.cc:2781] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:12.940050 2881 raft_consensus.cc:686] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 1 LEADER]: Becoming Leader. State: Replica: 9d7f7ffb3ee548faa1ef4834c1f4ee46, State: Running, Role: LEADER
I20240501 19:57:12.940285 2735 heartbeater.cc:503] Master 127.20.210.62:44939 was elected leader, sending a full tablet report...
I20240501 19:57:12.940340 2881 consensus_queue.cc:237] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38391 } }
I20240501 19:57:12.940447 2879 ts_tablet_manager.cc:1398] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240501 19:57:12.942852 2283 catalog_manager.cc:5484] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d7f7ffb3ee548faa1ef4834c1f4ee46 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38391 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:12.953236 21320 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240501 19:57:12.954336 21320 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 4b1f3070b19c49cb8cc3cce7e3d399db to finish bootstrapping
I20240501 19:57:12.958942 21320 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 1b01c2708530416c96980d5b57b68993 to finish bootstrapping
I20240501 19:57:12.963418 21320 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 9d7f7ffb3ee548faa1ef4834c1f4ee46 to finish bootstrapping
I20240501 19:57:12.967326 21320 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 3729e12454ae4d6a8048c13b825e2cfc to finish bootstrapping
I20240501 19:57:12.971704 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 2607
I20240501 19:57:12.976987 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 2738
I20240501 19:57:13.443249 2472 heartbeater.cc:503] Master 127.20.210.62:44939 was elected leader, sending a full tablet report...
I20240501 19:57:13.600090 2604 heartbeater.cc:503] Master 127.20.210.62:44939 was elected leader, sending a full tablet report...
I20240501 19:57:16.732944 2283 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48762:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:57:16.744067 2537 tablet_service.cc:1467] Processing CreateTablet for tablet 99ef8c82ee994020b21339f286523dd3 (DEFAULT_TABLE table=rebalance_test_table_0 [id=9483866ad0274e58aa4acdb0c711888e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:16.744922 2537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99ef8c82ee994020b21339f286523dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:16.744776 2405 tablet_service.cc:1467] Processing CreateTablet for tablet 19155fc130d44d65b8f0f7ba9820a906 (DEFAULT_TABLE table=rebalance_test_table_0 [id=9483866ad0274e58aa4acdb0c711888e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:16.744786 2404 tablet_service.cc:1467] Processing CreateTablet for tablet abbfb92322eb4d1da2ac56fef83ff692 (DEFAULT_TABLE table=rebalance_test_table_0 [id=9483866ad0274e58aa4acdb0c711888e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:16.745627 2404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abbfb92322eb4d1da2ac56fef83ff692. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:16.745862 2405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19155fc130d44d65b8f0f7ba9820a906. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:16.749873 2898 tablet_bootstrap.cc:492] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993: Bootstrap starting.
I20240501 19:57:16.750461 2899 tablet_bootstrap.cc:492] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db: Bootstrap starting.
I20240501 19:57:16.751677 2898 tablet_bootstrap.cc:654] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:16.751925 2899 tablet_bootstrap.cc:654] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:16.752426 2898 log.cc:824] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:16.752537 2899 log.cc:824] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:16.753847 2898 tablet_bootstrap.cc:492] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993: No bootstrap required, opened a new log
I20240501 19:57:16.753847 2899 tablet_bootstrap.cc:492] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db: No bootstrap required, opened a new log
I20240501 19:57:16.753996 2899 ts_tablet_manager.cc:1367] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 19:57:16.753996 2898 ts_tablet_manager.cc:1367] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 19:57:16.757287 2899 raft_consensus.cc:348] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:16.757280 2898 raft_consensus.cc:348] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:16.757454 2899 raft_consensus.cc:374] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:16.757454 2898 raft_consensus.cc:374] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:16.757503 2898 raft_consensus.cc:729] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b01c2708530416c96980d5b57b68993, State: Initialized, Role: FOLLOWER
I20240501 19:57:16.757503 2899 raft_consensus.cc:729] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b1f3070b19c49cb8cc3cce7e3d399db, State: Initialized, Role: FOLLOWER
I20240501 19:57:16.757913 2899 consensus_queue.cc:260] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:16.757913 2898 consensus_queue.cc:260] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:16.758031 2899 raft_consensus.cc:388] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:16.758031 2898 raft_consensus.cc:388] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:16.758090 2898 raft_consensus.cc:482] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:16.758086 2899 raft_consensus.cc:482] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:16.758217 2899 raft_consensus.cc:3037] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:16.758217 2898 raft_consensus.cc:3037] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:16.759080 2899 raft_consensus.cc:504] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:16.759080 2898 raft_consensus.cc:504] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:16.759443 2899 leader_election.cc:302] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b1f3070b19c49cb8cc3cce7e3d399db; no voters:
I20240501 19:57:16.759443 2898 leader_election.cc:302] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b01c2708530416c96980d5b57b68993; no voters:
I20240501 19:57:16.759696 2898 leader_election.cc:288] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:16.759697 2899 leader_election.cc:288] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:16.759766 2903 raft_consensus.cc:2781] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:16.759938 2904 raft_consensus.cc:2781] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:16.760387 2898 ts_tablet_manager.cc:1398] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20240501 19:57:16.760387 2899 ts_tablet_manager.cc:1398] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20240501 19:57:16.760517 2904 raft_consensus.cc:686] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 [term 1 LEADER]: Becoming Leader. State: Replica: 1b01c2708530416c96980d5b57b68993, State: Running, Role: LEADER
I20240501 19:57:16.760586 2899 tablet_bootstrap.cc:492] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db: Bootstrap starting.
I20240501 19:57:16.760622 2903 raft_consensus.cc:686] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 LEADER]: Becoming Leader. State: Replica: 4b1f3070b19c49cb8cc3cce7e3d399db, State: Running, Role: LEADER
I20240501 19:57:16.760851 2903 consensus_queue.cc:237] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:16.761335 2899 tablet_bootstrap.cc:654] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:16.762300 2899 tablet_bootstrap.cc:492] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db: No bootstrap required, opened a new log
I20240501 19:57:16.762367 2899 ts_tablet_manager.cc:1367] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:16.762200 2904 consensus_queue.cc:237] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:16.762795 2899 raft_consensus.cc:348] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:16.762862 2899 raft_consensus.cc:374] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:16.762892 2899 raft_consensus.cc:729] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b1f3070b19c49cb8cc3cce7e3d399db, State: Initialized, Role: FOLLOWER
I20240501 19:57:16.762969 2899 consensus_queue.cc:260] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:16.763036 2899 raft_consensus.cc:388] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:16.763062 2899 raft_consensus.cc:482] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:16.763095 2899 raft_consensus.cc:3037] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:16.763439 2283 catalog_manager.cc:5484] T 19155fc130d44d65b8f0f7ba9820a906 P 4b1f3070b19c49cb8cc3cce7e3d399db reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b1f3070b19c49cb8cc3cce7e3d399db (127.20.210.1). New cstate: current_term: 1 leader_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:16.763824 2899 raft_consensus.cc:504] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:16.763927 2899 leader_election.cc:302] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b1f3070b19c49cb8cc3cce7e3d399db; no voters:
I20240501 19:57:16.764019 2899 leader_election.cc:288] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:16.764099 2907 raft_consensus.cc:2781] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:16.764207 2899 ts_tablet_manager.cc:1398] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240501 19:57:16.764305 2907 raft_consensus.cc:686] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 LEADER]: Becoming Leader. State: Replica: 4b1f3070b19c49cb8cc3cce7e3d399db, State: Running, Role: LEADER
I20240501 19:57:16.764575 2907 consensus_queue.cc:237] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:16.764891 2282 catalog_manager.cc:5484] T 99ef8c82ee994020b21339f286523dd3 P 1b01c2708530416c96980d5b57b68993 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b01c2708530416c96980d5b57b68993 (127.20.210.2). New cstate: current_term: 1 leader_uuid: "1b01c2708530416c96980d5b57b68993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:16.767668 2282 catalog_manager.cc:5484] T abbfb92322eb4d1da2ac56fef83ff692 P 4b1f3070b19c49cb8cc3cce7e3d399db reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b1f3070b19c49cb8cc3cce7e3d399db (127.20.210.1). New cstate: current_term: 1 leader_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:16.923683 2282 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48762:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:57:16.928268 2537 tablet_service.cc:1467] Processing CreateTablet for tablet e002f4977f6e4acdb9213ac688be09a9 (DEFAULT_TABLE table=rebalance_test_table_1 [id=2ba5fa4c7c594e1b8a55b4921c6e087f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:16.928367 2536 tablet_service.cc:1467] Processing CreateTablet for tablet faf63a6e451b44f1bbd9d8582f259c2b (DEFAULT_TABLE table=rebalance_test_table_1 [id=2ba5fa4c7c594e1b8a55b4921c6e087f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:16.928413 2405 tablet_service.cc:1467] Processing CreateTablet for tablet f92935d8887c49e2833e4478374eed5d (DEFAULT_TABLE table=rebalance_test_table_1 [id=2ba5fa4c7c594e1b8a55b4921c6e087f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:16.928555 2537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e002f4977f6e4acdb9213ac688be09a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:16.928664 2405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f92935d8887c49e2833e4478374eed5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:16.928735 2536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faf63a6e451b44f1bbd9d8582f259c2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:16.930301 2898 tablet_bootstrap.cc:492] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993: Bootstrap starting.
I20240501 19:57:16.930670 2899 tablet_bootstrap.cc:492] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db: Bootstrap starting.
I20240501 19:57:16.931139 2898 tablet_bootstrap.cc:654] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:16.931514 2899 tablet_bootstrap.cc:654] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:16.932104 2898 tablet_bootstrap.cc:492] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993: No bootstrap required, opened a new log
I20240501 19:57:16.932189 2898 ts_tablet_manager.cc:1367] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:16.932513 2899 tablet_bootstrap.cc:492] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db: No bootstrap required, opened a new log
I20240501 19:57:16.932591 2899 ts_tablet_manager.cc:1367] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:16.932601 2898 raft_consensus.cc:348] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:16.932677 2898 raft_consensus.cc:374] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:16.932694 2898 raft_consensus.cc:729] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b01c2708530416c96980d5b57b68993, State: Initialized, Role: FOLLOWER
I20240501 19:57:16.932790 2898 consensus_queue.cc:260] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:16.932848 2898 raft_consensus.cc:388] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:16.932868 2898 raft_consensus.cc:482] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:16.932893 2898 raft_consensus.cc:3037] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:16.932968 2899 raft_consensus.cc:348] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:16.933034 2899 raft_consensus.cc:374] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:16.933053 2899 raft_consensus.cc:729] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b1f3070b19c49cb8cc3cce7e3d399db, State: Initialized, Role: FOLLOWER
I20240501 19:57:16.933133 2899 consensus_queue.cc:260] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:16.933174 2899 raft_consensus.cc:388] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:16.933192 2899 raft_consensus.cc:482] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:16.933216 2899 raft_consensus.cc:3037] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:16.933434 2898 raft_consensus.cc:504] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:16.933518 2898 leader_election.cc:302] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b01c2708530416c96980d5b57b68993; no voters:
I20240501 19:57:16.933578 2898 leader_election.cc:288] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:16.933660 2908 raft_consensus.cc:2781] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:16.933744 2908 raft_consensus.cc:686] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 [term 1 LEADER]: Becoming Leader. State: Replica: 1b01c2708530416c96980d5b57b68993, State: Running, Role: LEADER
I20240501 19:57:16.933840 2908 consensus_queue.cc:237] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:16.933912 2898 ts_tablet_manager.cc:1398] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:16.934022 2898 tablet_bootstrap.cc:492] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993: Bootstrap starting.
I20240501 19:57:16.934170 2899 raft_consensus.cc:504] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:16.934321 2899 leader_election.cc:302] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b1f3070b19c49cb8cc3cce7e3d399db; no voters:
I20240501 19:57:16.934396 2899 leader_election.cc:288] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:16.934649 2913 raft_consensus.cc:2781] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:16.934741 2913 raft_consensus.cc:686] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 LEADER]: Becoming Leader. State: Replica: 4b1f3070b19c49cb8cc3cce7e3d399db, State: Running, Role: LEADER
I20240501 19:57:16.934859 2913 consensus_queue.cc:237] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:16.935111 2898 tablet_bootstrap.cc:654] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:16.935307 2899 ts_tablet_manager.cc:1398] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:16.936148 2282 catalog_manager.cc:5484] T f92935d8887c49e2833e4478374eed5d P 4b1f3070b19c49cb8cc3cce7e3d399db reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b1f3070b19c49cb8cc3cce7e3d399db (127.20.210.1). New cstate: current_term: 1 leader_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:16.936235 2283 catalog_manager.cc:5484] T faf63a6e451b44f1bbd9d8582f259c2b P 1b01c2708530416c96980d5b57b68993 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b01c2708530416c96980d5b57b68993 (127.20.210.2). New cstate: current_term: 1 leader_uuid: "1b01c2708530416c96980d5b57b68993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:16.937117 2898 tablet_bootstrap.cc:492] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993: No bootstrap required, opened a new log
I20240501 19:57:16.937201 2898 ts_tablet_manager.cc:1367] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:16.937507 2898 raft_consensus.cc:348] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:16.937594 2898 raft_consensus.cc:374] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:16.937624 2898 raft_consensus.cc:729] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b01c2708530416c96980d5b57b68993, State: Initialized, Role: FOLLOWER
I20240501 19:57:16.937721 2898 consensus_queue.cc:260] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:16.937796 2898 raft_consensus.cc:388] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:16.937826 2898 raft_consensus.cc:482] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:16.937860 2898 raft_consensus.cc:3037] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:16.938517 2898 raft_consensus.cc:504] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:16.938604 2898 leader_election.cc:302] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b01c2708530416c96980d5b57b68993; no voters:
I20240501 19:57:16.938668 2898 leader_election.cc:288] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:16.938728 2908 raft_consensus.cc:2781] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:16.938798 2908 raft_consensus.cc:686] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 [term 1 LEADER]: Becoming Leader. State: Replica: 1b01c2708530416c96980d5b57b68993, State: Running, Role: LEADER
I20240501 19:57:16.938881 2898 ts_tablet_manager.cc:1398] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:16.938896 2908 consensus_queue.cc:237] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:16.940260 2283 catalog_manager.cc:5484] T e002f4977f6e4acdb9213ac688be09a9 P 1b01c2708530416c96980d5b57b68993 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b01c2708530416c96980d5b57b68993 (127.20.210.2). New cstate: current_term: 1 leader_uuid: "1b01c2708530416c96980d5b57b68993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:17.083609 2282 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48762:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:57:17.088572 2405 tablet_service.cc:1467] Processing CreateTablet for tablet 03ff1775e64844f7a92fee1a438315f8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=5a374baea5a143f6b76646b319a2cfdb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:17.088701 2537 tablet_service.cc:1467] Processing CreateTablet for tablet cecb70e571374137af8bc45c9f1b94ae (DEFAULT_TABLE table=rebalance_test_table_2 [id=5a374baea5a143f6b76646b319a2cfdb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:17.088744 2404 tablet_service.cc:1467] Processing CreateTablet for tablet e89c76f63ae049dd98e22f62e2eb3376 (DEFAULT_TABLE table=rebalance_test_table_2 [id=5a374baea5a143f6b76646b319a2cfdb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:17.088882 2405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03ff1775e64844f7a92fee1a438315f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:17.088918 2537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cecb70e571374137af8bc45c9f1b94ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:17.089040 2404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e89c76f63ae049dd98e22f62e2eb3376. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:17.090822 2899 tablet_bootstrap.cc:492] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db: Bootstrap starting.
I20240501 19:57:17.090909 2898 tablet_bootstrap.cc:492] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993: Bootstrap starting.
I20240501 19:57:17.091727 2899 tablet_bootstrap.cc:654] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:17.091729 2898 tablet_bootstrap.cc:654] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:17.092907 2899 tablet_bootstrap.cc:492] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db: No bootstrap required, opened a new log
I20240501 19:57:17.092984 2899 ts_tablet_manager.cc:1367] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:17.093113 2898 tablet_bootstrap.cc:492] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993: No bootstrap required, opened a new log
I20240501 19:57:17.093197 2898 ts_tablet_manager.cc:1367] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:17.093425 2899 raft_consensus.cc:348] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:17.093508 2899 raft_consensus.cc:374] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:17.093540 2899 raft_consensus.cc:729] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b1f3070b19c49cb8cc3cce7e3d399db, State: Initialized, Role: FOLLOWER
I20240501 19:57:17.093575 2898 raft_consensus.cc:348] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:17.093683 2898 raft_consensus.cc:374] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:17.093678 2899 consensus_queue.cc:260] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:17.093722 2898 raft_consensus.cc:729] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b01c2708530416c96980d5b57b68993, State: Initialized, Role: FOLLOWER
I20240501 19:57:17.093758 2899 raft_consensus.cc:388] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:17.093799 2899 raft_consensus.cc:482] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:17.093849 2899 raft_consensus.cc:3037] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:17.093830 2898 consensus_queue.cc:260] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:17.093892 2898 raft_consensus.cc:388] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:17.093937 2898 raft_consensus.cc:482] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:17.093979 2898 raft_consensus.cc:3037] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:17.094487 2899 raft_consensus.cc:504] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:17.094630 2899 leader_election.cc:302] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b1f3070b19c49cb8cc3cce7e3d399db; no voters:
I20240501 19:57:17.094652 2898 raft_consensus.cc:504] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:17.094725 2899 leader_election.cc:288] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:17.094785 2907 raft_consensus.cc:2781] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:17.094797 2898 leader_election.cc:302] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b01c2708530416c96980d5b57b68993; no voters:
I20240501 19:57:17.094851 2907 raft_consensus.cc:686] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 LEADER]: Becoming Leader. State: Replica: 4b1f3070b19c49cb8cc3cce7e3d399db, State: Running, Role: LEADER
I20240501 19:57:17.094878 2898 leader_election.cc:288] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:17.094962 2899 ts_tablet_manager.cc:1398] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:17.094977 2904 raft_consensus.cc:2781] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:17.094946 2907 consensus_queue.cc:237] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:17.095036 2904 raft_consensus.cc:686] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 [term 1 LEADER]: Becoming Leader. State: Replica: 1b01c2708530416c96980d5b57b68993, State: Running, Role: LEADER
I20240501 19:57:17.095116 2904 consensus_queue.cc:237] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:17.095162 2899 tablet_bootstrap.cc:492] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db: Bootstrap starting.
I20240501 19:57:17.096000 2899 tablet_bootstrap.cc:654] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:17.096355 2282 catalog_manager.cc:5484] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b01c2708530416c96980d5b57b68993 (127.20.210.2). New cstate: current_term: 1 leader_uuid: "1b01c2708530416c96980d5b57b68993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:17.097029 2283 catalog_manager.cc:5484] T e89c76f63ae049dd98e22f62e2eb3376 P 4b1f3070b19c49cb8cc3cce7e3d399db reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b1f3070b19c49cb8cc3cce7e3d399db (127.20.210.1). New cstate: current_term: 1 leader_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:17.097705 2898 ts_tablet_manager.cc:1398] T cecb70e571374137af8bc45c9f1b94ae P 1b01c2708530416c96980d5b57b68993: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20240501 19:57:17.098452 2899 tablet_bootstrap.cc:492] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db: No bootstrap required, opened a new log
I20240501 19:57:17.098541 2899 ts_tablet_manager.cc:1367] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:17.098989 2899 raft_consensus.cc:348] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:17.099079 2899 raft_consensus.cc:374] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:17.099113 2899 raft_consensus.cc:729] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b1f3070b19c49cb8cc3cce7e3d399db, State: Initialized, Role: FOLLOWER
I20240501 19:57:17.099226 2899 consensus_queue.cc:260] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:17.099326 2899 raft_consensus.cc:388] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:17.099367 2899 raft_consensus.cc:482] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:17.099414 2899 raft_consensus.cc:3037] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:17.100186 2899 raft_consensus.cc:504] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:17.100297 2899 leader_election.cc:302] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b1f3070b19c49cb8cc3cce7e3d399db; no voters:
I20240501 19:57:17.100386 2899 leader_election.cc:288] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:17.100427 2903 raft_consensus.cc:2781] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:17.100515 2903 raft_consensus.cc:686] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 LEADER]: Becoming Leader. State: Replica: 4b1f3070b19c49cb8cc3cce7e3d399db, State: Running, Role: LEADER
I20240501 19:57:17.100634 2899 ts_tablet_manager.cc:1398] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:17.100612 2903 consensus_queue.cc:237] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:17.102403 2283 catalog_manager.cc:5484] T 03ff1775e64844f7a92fee1a438315f8 P 4b1f3070b19c49cb8cc3cce7e3d399db reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b1f3070b19c49cb8cc3cce7e3d399db (127.20.210.1). New cstate: current_term: 1 leader_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:17.253451 2283 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48762:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240501 19:57:17.257784 2537 tablet_service.cc:1467] Processing CreateTablet for tablet c855e2672d4146e8a90dc4ff065d4f24 (DEFAULT_TABLE table=rebalance_test_table_3 [id=feb5895a112c4d169ef8aafe80cce86a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:17.257844 2536 tablet_service.cc:1467] Processing CreateTablet for tablet bbbe1f605fd2477bac7005d43bb164de (DEFAULT_TABLE table=rebalance_test_table_3 [id=feb5895a112c4d169ef8aafe80cce86a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:17.257946 2405 tablet_service.cc:1467] Processing CreateTablet for tablet e6a5e1c705bf4e1b892d7fe631daf4c2 (DEFAULT_TABLE table=rebalance_test_table_3 [id=feb5895a112c4d169ef8aafe80cce86a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:17.258050 2537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c855e2672d4146e8a90dc4ff065d4f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:17.258211 2405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6a5e1c705bf4e1b892d7fe631daf4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:17.258245 2536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbbe1f605fd2477bac7005d43bb164de. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:17.259709 2898 tablet_bootstrap.cc:492] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993: Bootstrap starting.
I20240501 19:57:17.260128 2899 tablet_bootstrap.cc:492] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db: Bootstrap starting.
I20240501 19:57:17.260424 2898 tablet_bootstrap.cc:654] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:17.260951 2899 tablet_bootstrap.cc:654] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:17.261318 2898 tablet_bootstrap.cc:492] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993: No bootstrap required, opened a new log
I20240501 19:57:17.261379 2898 ts_tablet_manager.cc:1367] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:57:17.261739 2898 raft_consensus.cc:348] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:17.261812 2898 raft_consensus.cc:374] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:17.261830 2898 raft_consensus.cc:729] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b01c2708530416c96980d5b57b68993, State: Initialized, Role: FOLLOWER
I20240501 19:57:17.261924 2898 consensus_queue.cc:260] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:17.261957 2899 tablet_bootstrap.cc:492] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db: No bootstrap required, opened a new log
I20240501 19:57:17.261984 2898 raft_consensus.cc:388] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:17.262003 2898 raft_consensus.cc:482] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:17.262027 2899 ts_tablet_manager.cc:1367] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:17.262044 2898 raft_consensus.cc:3037] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:17.262423 2899 raft_consensus.cc:348] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:17.262506 2899 raft_consensus.cc:374] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:17.262539 2899 raft_consensus.cc:729] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b1f3070b19c49cb8cc3cce7e3d399db, State: Initialized, Role: FOLLOWER
I20240501 19:57:17.262645 2899 consensus_queue.cc:260] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:17.262733 2899 raft_consensus.cc:388] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:17.262778 2899 raft_consensus.cc:482] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:17.262818 2899 raft_consensus.cc:3037] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:17.262804 2898 raft_consensus.cc:504] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:17.262920 2898 leader_election.cc:302] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b01c2708530416c96980d5b57b68993; no voters:
I20240501 19:57:17.263000 2898 leader_election.cc:288] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:17.263104 2906 raft_consensus.cc:2781] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:17.263177 2906 raft_consensus.cc:686] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 [term 1 LEADER]: Becoming Leader. State: Replica: 1b01c2708530416c96980d5b57b68993, State: Running, Role: LEADER
I20240501 19:57:17.263320 2898 ts_tablet_manager.cc:1398] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20240501 19:57:17.263278 2906 consensus_queue.cc:237] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:17.263432 2898 tablet_bootstrap.cc:492] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993: Bootstrap starting.
I20240501 19:57:17.263504 2899 raft_consensus.cc:504] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:17.263614 2899 leader_election.cc:302] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b1f3070b19c49cb8cc3cce7e3d399db; no voters:
I20240501 19:57:17.263801 2907 raft_consensus.cc:2781] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:17.263886 2907 raft_consensus.cc:686] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db [term 1 LEADER]: Becoming Leader. State: Replica: 4b1f3070b19c49cb8cc3cce7e3d399db, State: Running, Role: LEADER
I20240501 19:57:17.263993 2907 consensus_queue.cc:237] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } }
I20240501 19:57:17.264343 2898 tablet_bootstrap.cc:654] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:17.264675 2899 leader_election.cc:288] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:17.264955 2899 ts_tablet_manager.cc:1398] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:17.265095 2283 catalog_manager.cc:5484] T c855e2672d4146e8a90dc4ff065d4f24 P 1b01c2708530416c96980d5b57b68993 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b01c2708530416c96980d5b57b68993 (127.20.210.2). New cstate: current_term: 1 leader_uuid: "1b01c2708530416c96980d5b57b68993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:17.265297 2282 catalog_manager.cc:5484] T e6a5e1c705bf4e1b892d7fe631daf4c2 P 4b1f3070b19c49cb8cc3cce7e3d399db reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b1f3070b19c49cb8cc3cce7e3d399db (127.20.210.1). New cstate: current_term: 1 leader_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1f3070b19c49cb8cc3cce7e3d399db" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 34267 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:17.265585 2898 tablet_bootstrap.cc:492] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993: No bootstrap required, opened a new log
I20240501 19:57:17.265645 2898 ts_tablet_manager.cc:1367] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:17.265987 2898 raft_consensus.cc:348] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:17.266067 2898 raft_consensus.cc:374] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:17.266093 2898 raft_consensus.cc:729] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b01c2708530416c96980d5b57b68993, State: Initialized, Role: FOLLOWER
I20240501 19:57:17.266183 2898 consensus_queue.cc:260] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:17.266286 2898 raft_consensus.cc:388] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:17.266324 2898 raft_consensus.cc:482] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:17.266359 2898 raft_consensus.cc:3037] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:17.267030 2898 raft_consensus.cc:504] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:17.267130 2898 leader_election.cc:302] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b01c2708530416c96980d5b57b68993; no voters:
I20240501 19:57:17.267201 2898 leader_election.cc:288] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:17.267413 2898 ts_tablet_manager.cc:1398] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:17.267536 2904 raft_consensus.cc:2781] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:17.267603 2904 raft_consensus.cc:686] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 [term 1 LEADER]: Becoming Leader. State: Replica: 1b01c2708530416c96980d5b57b68993, State: Running, Role: LEADER
I20240501 19:57:17.267714 2904 consensus_queue.cc:237] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } }
I20240501 19:57:17.269107 2281 catalog_manager.cc:5484] T bbbe1f605fd2477bac7005d43bb164de P 1b01c2708530416c96980d5b57b68993 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b01c2708530416c96980d5b57b68993 (127.20.210.2). New cstate: current_term: 1 leader_uuid: "1b01c2708530416c96980d5b57b68993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b01c2708530416c96980d5b57b68993" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 32961 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:17.410543 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.3:38391
--local_ip_for_outbound_sockets=127.20.210.3
--tserver_master_addrs=127.20.210.62:44939
--webserver_port=34947
--webserver_interface=127.20.210.3
--builtin_ntp_servers=127.20.210.20:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 19:57:17.510344 3008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:17.510514 3008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:17.510558 3008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:17.513338 3008 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:17.513391 3008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:17.513449 3008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.3
I20240501 19:57:17.516824 3008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.3:38391
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.20.210.3
--webserver_port=34947
--tserver_master_addrs=127.20.210.62:44939
--never_fsync=true
--heap_profile_path=/tmp/kudu.3008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.3
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:17.517531 3008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:17.518529 3008 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:17.526063 3013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:17.526836 3014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:17.527647 3016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:17.528437 3008 server_base.cc:1034] running on GCE node
I20240501 19:57:17.528743 3008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:17.529232 3008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:17.530372 3008 hybrid_clock.cc:648] HybridClock initialized: now 1714593437530363 us; error 27 us; skew 500 ppm
I20240501 19:57:17.531847 3008 webserver.cc:460] Webserver started at http://127.20.210.3:34947/ using document root <none> and password file <none>
I20240501 19:57:17.532339 3008 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:17.532409 3008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:17.535272 3008 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 19:57:17.536924 3022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:17.537811 3008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:17.537895 3008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:17.538199 3008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:17.563407 3008 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:17.563868 3008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:17.563980 3008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:17.564371 3008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:17.565337 3029 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240501 19:57:17.567988 3008 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240501 19:57:17.568053 3008 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20240501 19:57:17.568111 3008 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240501 19:57:17.569387 3008 ts_tablet_manager.cc:590] Registered 1 tablets
I20240501 19:57:17.569428 3008 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:17.569669 3029 tablet_bootstrap.cc:492] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46: Bootstrap starting.
I20240501 19:57:17.578083 3008 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.3:38391
I20240501 19:57:17.578158 3137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.3:38391 every 8 connection(s)
I20240501 19:57:17.579005 3008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 19:57:17.582856 3029 log.cc:824] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:17.585101 3029 tablet_bootstrap.cc:492] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:17.585410 3029 tablet_bootstrap.cc:492] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46: Bootstrap complete.
I20240501 19:57:17.586081 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 3008
I20240501 19:57:17.586614 3029 ts_tablet_manager.cc:1367] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.003s
I20240501 19:57:17.586609 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.4:44677
--local_ip_for_outbound_sockets=127.20.210.4
--tserver_master_addrs=127.20.210.62:44939
--webserver_port=44651
--webserver_interface=127.20.210.4
--builtin_ntp_servers=127.20.210.20:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:17.587148 3138 heartbeater.cc:347] Connected to a master server at 127.20.210.62:44939
I20240501 19:57:17.587340 3138 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:17.587810 3138 heartbeater.cc:511] Master 127.20.210.62:44939 requested a full tablet report, sending...
I20240501 19:57:17.589634 2281 ts_manager.cc:194] Re-registered known tserver with Master: 9d7f7ffb3ee548faa1ef4834c1f4ee46 (127.20.210.3:38391)
I20240501 19:57:17.589581 3029 raft_consensus.cc:348] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38391 } }
I20240501 19:57:17.589931 3029 raft_consensus.cc:729] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7f7ffb3ee548faa1ef4834c1f4ee46, State: Initialized, Role: FOLLOWER
I20240501 19:57:17.590361 3029 consensus_queue.cc:260] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38391 } }
I20240501 19:57:17.590472 3029 raft_consensus.cc:388] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:17.590515 3029 raft_consensus.cc:482] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:17.590552 2281 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.3:40895
I20240501 19:57:17.590597 3029 raft_consensus.cc:3037] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:17.591925 3029 raft_consensus.cc:504] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38391 } }
I20240501 19:57:17.592156 3029 leader_election.cc:302] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d7f7ffb3ee548faa1ef4834c1f4ee46; no voters:
I20240501 19:57:17.592346 3029 leader_election.cc:288] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [CANDIDATE]: Term 2 election: Requested vote from peers
I20240501 19:57:17.592425 3143 raft_consensus.cc:2781] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 19:57:17.592552 3143 raft_consensus.cc:686] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [term 2 LEADER]: Becoming Leader. State: Replica: 9d7f7ffb3ee548faa1ef4834c1f4ee46, State: Running, Role: LEADER
I20240501 19:57:17.593070 3029 ts_tablet_manager.cc:1398] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20240501 19:57:17.593021 3143 consensus_queue.cc:237] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38391 } }
I20240501 19:57:17.593384 3138 heartbeater.cc:503] Master 127.20.210.62:44939 was elected leader, sending a full tablet report...
I20240501 19:57:17.594727 2281 catalog_manager.cc:5484] T 7c4bce39c03a4c5191bc8a43612ac8e6 P 9d7f7ffb3ee548faa1ef4834c1f4ee46 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d7f7ffb3ee548faa1ef4834c1f4ee46" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38391 } health_report { overall_health: HEALTHY } } }
W20240501 19:57:17.685588 3142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:17.685745 3142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:17.685779 3142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:17.688393 3142 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:17.688445 3142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:17.688508 3142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.4
I20240501 19:57:17.691638 3142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.4:44677
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.20.210.4
--webserver_port=44651
--tserver_master_addrs=127.20.210.62:44939
--never_fsync=true
--heap_profile_path=/tmp/kudu.3142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.4
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:17.692288 3142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:17.693173 3142 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:17.700348 3155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:17.701094 3152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:17.701733 3153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:17.702430 3142 server_base.cc:1034] running on GCE node
I20240501 19:57:17.702734 3142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:17.703193 3142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:17.704341 3142 hybrid_clock.cc:648] HybridClock initialized: now 1714593437704329 us; error 22 us; skew 500 ppm
I20240501 19:57:17.705895 3142 webserver.cc:460] Webserver started at http://127.20.210.4:44651/ using document root <none> and password file <none>
I20240501 19:57:17.706426 3142 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:17.706497 3142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:17.709461 3142 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:17.711025 3161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:17.711869 3142 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:17.711967 3142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "3729e12454ae4d6a8048c13b825e2cfc"
format_stamp: "Formatted at 2024-05-01 19:57:12 on dist-test-slave-vgh4"
I20240501 19:57:17.712289 3142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:17.727694 3142 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:17.728123 3142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:17.728272 3142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:17.728699 3142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:17.729424 3142 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:17.729482 3142 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:17.729529 3142 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:17.729557 3142 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:17.736991 3142 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.4:44677
I20240501 19:57:17.737011 3275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.4:44677 every 8 connection(s)
I20240501 19:57:17.737772 3142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 19:57:17.744509 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 3142
I20240501 19:57:17.744985 3276 heartbeater.cc:347] Connected to a master server at 127.20.210.62:44939
I20240501 19:57:17.745179 3276 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:17.745733 3276 heartbeater.cc:511] Master 127.20.210.62:44939 requested a full tablet report, sending...
I20240501 19:57:17.746835 2281 ts_manager.cc:194] Re-registered known tserver with Master: 3729e12454ae4d6a8048c13b825e2cfc (127.20.210.4:44677)
I20240501 19:57:17.747422 2281 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.4:52259
I20240501 19:57:17.908457 2536 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:17.910329 2405 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:17.911513 3209 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:17.919930 3054 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:18.014711 2536 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:18.014714 3054 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:18.015353 3209 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:18.016355 2405 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:18.095052 3209 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:18.095240 2405 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:18.095620 3054 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:18.095835 2536 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
1f4f92ff7412455581cdfc3f66658434 | 127.20.210.62:44939 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.20.210.20:39273 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
1b01c2708530416c96980d5b57b68993 | 127.20.210.2:32961 | HEALTHY | <none> | 6 | 0
3729e12454ae4d6a8048c13b825e2cfc | 127.20.210.4:44677 | HEALTHY | <none> | 0 | 0
4b1f3070b19c49cb8cc3cce7e3d399db | 127.20.210.1:34267 | HEALTHY | <none> | 6 | 0
9d7f7ffb3ee548faa1ef4834c1f4ee46 | 127.20.210.3:38391 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 4 server(s) checked
local_ip_for_outbound_sockets | 127.20.210.1 | experimental | 127.20.210.1:34267
local_ip_for_outbound_sockets | 127.20.210.2 | experimental | 127.20.210.2:32961
local_ip_for_outbound_sockets | 127.20.210.3 | experimental | 127.20.210.3:38391
local_ip_for_outbound_sockets | 127.20.210.4 | experimental | 127.20.210.4:44677
never_fsync | true | unsafe,advanced | all 4 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 4 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 4 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
server_dump_info_format | pb | hidden | all 4 server(s) checked
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.20.210.1:34267
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.20.210.2:32961
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.20.210.3:38391
server_dump_info_path | /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.20.210.4:44677
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.20.210.20:39273 | all 4 server(s) checked
time_source | builtin | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 6
Third Quartile | 6
Maximum | 6
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 5
Tablets | 13
Replicas | 13
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240501 19:57:18.120762 21320 log_verifier.cc:126] Checking tablet 03ff1775e64844f7a92fee1a438315f8
I20240501 19:57:18.128129 21320 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 03ff1775e64844f7a92fee1a438315f8
I20240501 19:57:18.128175 21320 log_verifier.cc:126] Checking tablet 19155fc130d44d65b8f0f7ba9820a906
I20240501 19:57:18.134972 21320 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 19155fc130d44d65b8f0f7ba9820a906
I20240501 19:57:18.135015 21320 log_verifier.cc:126] Checking tablet 7c4bce39c03a4c5191bc8a43612ac8e6
I20240501 19:57:18.141633 21320 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 7c4bce39c03a4c5191bc8a43612ac8e6
I20240501 19:57:18.141672 21320 log_verifier.cc:126] Checking tablet 99ef8c82ee994020b21339f286523dd3
I20240501 19:57:18.150084 21320 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 99ef8c82ee994020b21339f286523dd3
I20240501 19:57:18.150130 21320 log_verifier.cc:126] Checking tablet abbfb92322eb4d1da2ac56fef83ff692
I20240501 19:57:18.157699 21320 log_verifier.cc:177] Verified matching terms for 1 ops in tablet abbfb92322eb4d1da2ac56fef83ff692
I20240501 19:57:18.157747 21320 log_verifier.cc:126] Checking tablet bbbe1f605fd2477bac7005d43bb164de
I20240501 19:57:18.164418 21320 log_verifier.cc:177] Verified matching terms for 1 ops in tablet bbbe1f605fd2477bac7005d43bb164de
I20240501 19:57:18.164460 21320 log_verifier.cc:126] Checking tablet c855e2672d4146e8a90dc4ff065d4f24
I20240501 19:57:18.171393 21320 log_verifier.cc:177] Verified matching terms for 2 ops in tablet c855e2672d4146e8a90dc4ff065d4f24
I20240501 19:57:18.171432 21320 log_verifier.cc:126] Checking tablet cecb70e571374137af8bc45c9f1b94ae
I20240501 19:57:18.178146 21320 log_verifier.cc:177] Verified matching terms for 1 ops in tablet cecb70e571374137af8bc45c9f1b94ae
I20240501 19:57:18.178190 21320 log_verifier.cc:126] Checking tablet e002f4977f6e4acdb9213ac688be09a9
I20240501 19:57:18.185150 21320 log_verifier.cc:177] Verified matching terms for 2 ops in tablet e002f4977f6e4acdb9213ac688be09a9
I20240501 19:57:18.185194 21320 log_verifier.cc:126] Checking tablet e6a5e1c705bf4e1b892d7fe631daf4c2
I20240501 19:57:18.191795 21320 log_verifier.cc:177] Verified matching terms for 1 ops in tablet e6a5e1c705bf4e1b892d7fe631daf4c2
I20240501 19:57:18.191833 21320 log_verifier.cc:126] Checking tablet e89c76f63ae049dd98e22f62e2eb3376
I20240501 19:57:18.198334 21320 log_verifier.cc:177] Verified matching terms for 1 ops in tablet e89c76f63ae049dd98e22f62e2eb3376
I20240501 19:57:18.198375 21320 log_verifier.cc:126] Checking tablet f92935d8887c49e2833e4478374eed5d
I20240501 19:57:18.205118 21320 log_verifier.cc:177] Verified matching terms for 1 ops in tablet f92935d8887c49e2833e4478374eed5d
I20240501 19:57:18.205160 21320 log_verifier.cc:126] Checking tablet faf63a6e451b44f1bbd9d8582f259c2b
I20240501 19:57:18.212014 21320 log_verifier.cc:177] Verified matching terms for 1 ops in tablet faf63a6e451b44f1bbd9d8582f259c2b
I20240501 19:57:18.212404 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 2343
I20240501 19:57:18.218099 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 2475
I20240501 19:57:18.223874 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 3008
I20240501 19:57:18.229084 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 3142
I20240501 19:57:18.234037 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 2251
2024-05-01T19:57:18Z chronyd exiting
[ OK ] RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove/5 (6230 ms)
[----------] 1 test from RebalancerAndSingleReplicaTablets (6230 ms total)
[----------] 2 tests from LocationAwareRebalancingParamTest
[ RUN ] LocationAwareRebalancingParamTest.Rebalance/idx7_rf2_t3_l_1_2_1
I20240501 19:57:18.264712 21320 test_util.cc:268] Using random seed: -455971451
I20240501 19:57:18.265349 21320 ts_itest-base.cc:112] Starting cluster with:
I20240501 19:57:18.265380 21320 ts_itest-base.cc:113] --------------
I20240501 19:57:18.265390 21320 ts_itest-base.cc:114] 4 tablet servers
I20240501 19:57:18.265400 21320 ts_itest-base.cc:115] 2 replicas per TS
I20240501 19:57:18.265421 21320 ts_itest-base.cc:116] --------------
2024-05-01T19:57:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T19:57:18Z Disabled control of system clock
I20240501 19:57:18.272964 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:40877
--webserver_interface=127.20.210.62
--webserver_port=0
--builtin_ntp_servers=127.20.210.20:34645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.210.62:40877
--location_mapping_cmd=/tmp/dist-test-taskNtLy6c/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/location-assignment.state --map /A:1 --map /B:2 --map /C:1
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false
--master_client_location_assignment_enabled=false
--allow_unsafe_replication_factor with env {}
W20240501 19:57:18.364527 3384 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240501 19:57:18.364712 3384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:18.364735 3384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:18.364775 3384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:18.367398 3384 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:18.367452 3384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 19:57:18.367473 3384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:18.367488 3384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 19:57:18.367502 3384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 19:57:18.370592 3384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:34645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--location_mapping_cmd=/tmp/dist-test-taskNtLy6c/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/location-assignment.state --map /A:1 --map /B:2 --map /C:1
--ipki_ca_key_size=768
--master_addresses=127.20.210.62:40877
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:40877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.20.210.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.3384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:18.371169 3384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:18.371973 3384 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:18.378196 3389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:18.379041 3390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:18.379791 3392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:18.380506 3384 server_base.cc:1034] running on GCE node
I20240501 19:57:18.380909 3384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:18.381637 3384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:18.382776 3384 hybrid_clock.cc:648] HybridClock initialized: now 1714593438382766 us; error 22 us; skew 500 ppm
I20240501 19:57:18.384516 3384 webserver.cc:460] Webserver started at http://127.20.210.62:41741/ using document root <none> and password file <none>
I20240501 19:57:18.385001 3384 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:18.385061 3384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:18.385212 3384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:18.386616 3384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "254c5f306cd74c38afc040766cda973c"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:18.387022 3384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "254c5f306cd74c38afc040766cda973c"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:18.389914 3384 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 19:57:18.391774 3398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:18.392560 3384 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:18.392657 3384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
uuid: "254c5f306cd74c38afc040766cda973c"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:18.392730 3384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:18.410290 3384 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:18.410784 3384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:18.410916 3384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:18.417176 3384 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.62:40877
I20240501 19:57:18.417188 3451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.62:40877 every 8 connection(s)
I20240501 19:57:18.417939 3384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 19:57:18.420130 3452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:18.424129 3452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c: Bootstrap starting.
I20240501 19:57:18.426005 3452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:18.426586 3452 log.cc:824] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:18.427101 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 3384
I20240501 19:57:18.427240 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 19:57:18.428088 3452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c: No bootstrap required, opened a new log
I20240501 19:57:18.431185 3452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254c5f306cd74c38afc040766cda973c" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 40877 } }
I20240501 19:57:18.431375 3452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:18.431427 3452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 254c5f306cd74c38afc040766cda973c, State: Initialized, Role: FOLLOWER
I20240501 19:57:18.431803 3452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254c5f306cd74c38afc040766cda973c" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 40877 } }
I20240501 19:57:18.431919 3452 raft_consensus.cc:388] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:18.431974 3452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:18.432075 3452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:18.432708 3452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254c5f306cd74c38afc040766cda973c" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 40877 } }
I20240501 19:57:18.432999 3452 leader_election.cc:302] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 254c5f306cd74c38afc040766cda973c; no voters:
I20240501 19:57:18.433230 3452 leader_election.cc:288] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:18.433549 3457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:18.433729 3457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [term 1 LEADER]: Becoming Leader. State: Replica: 254c5f306cd74c38afc040766cda973c, State: Running, Role: LEADER
I20240501 19:57:18.434000 3457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254c5f306cd74c38afc040766cda973c" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 40877 } }
I20240501 19:57:18.434139 3452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [sys.catalog]: configured and running, proceeding with master startup.
I20240501 19:57:18.435362 3458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "254c5f306cd74c38afc040766cda973c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254c5f306cd74c38afc040766cda973c" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 40877 } } }
I20240501 19:57:18.435390 3459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 254c5f306cd74c38afc040766cda973c. Latest consensus state: current_term: 1 leader_uuid: "254c5f306cd74c38afc040766cda973c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "254c5f306cd74c38afc040766cda973c" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 40877 } } }
I20240501 19:57:18.435556 3458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [sys.catalog]: This master's current role is: LEADER
I20240501 19:57:18.435845 3459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c [sys.catalog]: This master's current role is: LEADER
I20240501 19:57:18.436100 3468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 19:57:18.438769 3468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 19:57:18.442585 3468 catalog_manager.cc:1347] Generated new cluster ID: e4beaf823e41458f889531574584d929
I20240501 19:57:18.442639 3468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 19:57:18.452584 3468 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 19:57:18.453258 3468 catalog_manager.cc:1504] Loading token signing keys...
I20240501 19:57:18.458599 3468 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 254c5f306cd74c38afc040766cda973c: Generated new TSK 0
I20240501 19:57:18.459121 3468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 19:57:18.463745 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.1:0
--local_ip_for_outbound_sockets=127.20.210.1
--webserver_interface=127.20.210.1
--webserver_port=0
--tserver_master_addrs=127.20.210.62:40877
--builtin_ntp_servers=127.20.210.20:34645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 19:57:18.556936 3476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:18.557123 3476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:18.557191 3476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:18.560000 3476 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:18.560060 3476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:18.560154 3476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.1
I20240501 19:57:18.563225 3476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:34645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.20.210.1
--webserver_port=0
--tserver_master_addrs=127.20.210.62:40877
--never_fsync=true
--heap_profile_path=/tmp/kudu.3476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.1
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:18.563905 3476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:18.564867 3476 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:18.572767 3484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:18.573294 3481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:18.574038 3482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:18.574826 3476 server_base.cc:1034] running on GCE node
I20240501 19:57:18.575145 3476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:18.575649 3476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:18.576808 3476 hybrid_clock.cc:648] HybridClock initialized: now 1714593438576790 us; error 36 us; skew 500 ppm
I20240501 19:57:18.578517 3476 webserver.cc:460] Webserver started at http://127.20.210.1:35733/ using document root <none> and password file <none>
I20240501 19:57:18.579020 3476 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:18.579072 3476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:18.579227 3476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:18.580698 3476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "af80abcc7a194cc582c7c45a62e49086"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:18.581127 3476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "af80abcc7a194cc582c7c45a62e49086"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:18.584120 3476 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 19:57:18.585996 3490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:18.586895 3476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:18.586999 3476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "af80abcc7a194cc582c7c45a62e49086"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:18.587093 3476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:18.613523 3476 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:18.614001 3476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:18.614130 3476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:18.614594 3476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:18.615324 3476 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:18.615365 3476 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:18.615399 3476 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:18.615428 3476 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:18.622519 3476 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.1:36143
I20240501 19:57:18.622570 3604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.1:36143 every 8 connection(s)
I20240501 19:57:18.623267 3476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 19:57:18.629168 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 3476
I20240501 19:57:18.629312 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 19:57:18.630790 3605 heartbeater.cc:347] Connected to a master server at 127.20.210.62:40877
I20240501 19:57:18.630975 3605 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:18.631286 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.2:0
--local_ip_for_outbound_sockets=127.20.210.2
--webserver_interface=127.20.210.2
--webserver_port=0
--tserver_master_addrs=127.20.210.62:40877
--builtin_ntp_servers=127.20.210.20:34645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:18.631422 3605 heartbeater.cc:511] Master 127.20.210.62:40877 requested a full tablet report, sending...
I20240501 19:57:18.668319 3415 ts_manager.cc:194] Registered new tserver with Master: af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143)
I20240501 19:57:18.669626 3415 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.1:56345
W20240501 19:57:18.725642 3608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:18.725800 3608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:18.725844 3608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:18.728633 3608 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:18.728688 3608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:18.728778 3608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.2
I20240501 19:57:18.731932 3608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:34645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.20.210.2
--webserver_port=0
--tserver_master_addrs=127.20.210.62:40877
--never_fsync=true
--heap_profile_path=/tmp/kudu.3608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.2
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:18.732563 3608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:18.733402 3608 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:18.740469 3617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:18.741003 3615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:18.741824 3614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:18.742507 3608 server_base.cc:1034] running on GCE node
I20240501 19:57:18.742816 3608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:18.743256 3608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:18.744396 3608 hybrid_clock.cc:648] HybridClock initialized: now 1714593438744375 us; error 36 us; skew 500 ppm
I20240501 19:57:18.746058 3608 webserver.cc:460] Webserver started at http://127.20.210.2:36663/ using document root <none> and password file <none>
I20240501 19:57:18.746567 3608 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:18.746624 3608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:18.746783 3608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:18.748163 3608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "c1a344780dbb4de9bcb624ac5d91d469"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:18.748608 3608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "c1a344780dbb4de9bcb624ac5d91d469"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:18.751546 3608 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 19:57:18.753237 3623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:18.754036 3608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:18.754124 3608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "c1a344780dbb4de9bcb624ac5d91d469"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:18.754194 3608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:18.766929 3608 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:18.767344 3608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:18.767478 3608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:18.767887 3608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:18.768625 3608 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:18.768671 3608 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:18.768728 3608 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:18.768760 3608 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:18.776016 3608 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.2:34623
I20240501 19:57:18.776060 3737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.2:34623 every 8 connection(s)
I20240501 19:57:18.776751 3608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 19:57:18.784530 3738 heartbeater.cc:347] Connected to a master server at 127.20.210.62:40877
I20240501 19:57:18.784710 3738 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:18.785141 3738 heartbeater.cc:511] Master 127.20.210.62:40877 requested a full tablet report, sending...
I20240501 19:57:18.786746 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 3608
I20240501 19:57:18.786851 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 19:57:18.789840 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.3:0
--local_ip_for_outbound_sockets=127.20.210.3
--webserver_interface=127.20.210.3
--webserver_port=0
--tserver_master_addrs=127.20.210.62:40877
--builtin_ntp_servers=127.20.210.20:34645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:18.816704 3415 ts_manager.cc:194] Registered new tserver with Master: c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2:34623)
I20240501 19:57:18.817277 3415 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.2:52919
W20240501 19:57:18.886986 3742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:18.887153 3742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:18.887214 3742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:18.889946 3742 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:18.890007 3742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:18.890093 3742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.3
I20240501 19:57:18.893322 3742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:34645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.20.210.3
--webserver_port=0
--tserver_master_addrs=127.20.210.62:40877
--never_fsync=true
--heap_profile_path=/tmp/kudu.3742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.3
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:18.893976 3742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:18.894905 3742 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:18.902518 3748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:18.903070 3750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:18.903869 3747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:18.904593 3742 server_base.cc:1034] running on GCE node
I20240501 19:57:18.904898 3742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:18.905346 3742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:18.906483 3742 hybrid_clock.cc:648] HybridClock initialized: now 1714593438906478 us; error 23 us; skew 500 ppm
I20240501 19:57:18.908046 3742 webserver.cc:460] Webserver started at http://127.20.210.3:41613/ using document root <none> and password file <none>
I20240501 19:57:18.908535 3742 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:18.908596 3742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:18.908756 3742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:18.910219 3742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "f0f5056255ed4763b222e74191d4d582"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:18.910663 3742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "f0f5056255ed4763b222e74191d4d582"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:18.913424 3742 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:18.915225 3756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:18.916024 3742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:18.916121 3742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "f0f5056255ed4763b222e74191d4d582"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:18.916193 3742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:18.940054 3742 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:18.940506 3742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:18.940641 3742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:18.941068 3742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:18.941803 3742 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:18.941851 3742 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:18.941905 3742 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:18.941934 3742 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:18.950011 3742 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.3:38243
I20240501 19:57:18.950034 3870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.3:38243 every 8 connection(s)
I20240501 19:57:18.950835 3742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 19:57:18.957391 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 3742
I20240501 19:57:18.957491 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 19:57:18.957950 3871 heartbeater.cc:347] Connected to a master server at 127.20.210.62:40877
I20240501 19:57:18.958132 3871 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:18.958629 3871 heartbeater.cc:511] Master 127.20.210.62:40877 requested a full tablet report, sending...
I20240501 19:57:18.959652 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.4:0
--local_ip_for_outbound_sockets=127.20.210.4
--webserver_interface=127.20.210.4
--webserver_port=0
--tserver_master_addrs=127.20.210.62:40877
--builtin_ntp_servers=127.20.210.20:34645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:18.989910 3415 ts_manager.cc:194] Registered new tserver with Master: f0f5056255ed4763b222e74191d4d582 (127.20.210.3:38243)
I20240501 19:57:18.990571 3415 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.3:43861
W20240501 19:57:19.053858 3874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:19.054009 3874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:19.054046 3874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:19.056612 3874 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:19.056660 3874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:19.056721 3874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.4
I20240501 19:57:19.059949 3874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:34645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.4:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.20.210.4
--webserver_port=0
--tserver_master_addrs=127.20.210.62:40877
--never_fsync=true
--heap_profile_path=/tmp/kudu.3874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.4
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:19.060550 3874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:19.061378 3874 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:19.069036 3883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:19.069531 3880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:19.070333 3881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:19.071115 3874 server_base.cc:1034] running on GCE node
I20240501 19:57:19.071405 3874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:19.071848 3874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:19.073001 3874 hybrid_clock.cc:648] HybridClock initialized: now 1714593439072981 us; error 36 us; skew 500 ppm
I20240501 19:57:19.074568 3874 webserver.cc:460] Webserver started at http://127.20.210.4:43371/ using document root <none> and password file <none>
I20240501 19:57:19.075055 3874 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:19.075114 3874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:19.075273 3874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:19.076579 3874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "1d5d277450784129ab6e7b31597a48d5"
format_stamp: "Formatted at 2024-05-01 19:57:19 on dist-test-slave-vgh4"
I20240501 19:57:19.077016 3874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "1d5d277450784129ab6e7b31597a48d5"
format_stamp: "Formatted at 2024-05-01 19:57:19 on dist-test-slave-vgh4"
I20240501 19:57:19.079865 3874 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 19:57:19.081626 3889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:19.082551 3874 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:19.082643 3874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "1d5d277450784129ab6e7b31597a48d5"
format_stamp: "Formatted at 2024-05-01 19:57:19 on dist-test-slave-vgh4"
I20240501 19:57:19.082731 3874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:19.119836 3874 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:19.120273 3874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:19.120402 3874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:19.120813 3874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:19.121554 3874 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:19.121601 3874 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:19.121656 3874 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:19.121683 3874 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:19.129125 3874 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.4:42263
I20240501 19:57:19.129175 4003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.4:42263 every 8 connection(s)
I20240501 19:57:19.129920 3874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 19:57:19.135015 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 3874
I20240501 19:57:19.135134 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 19:57:19.137496 4004 heartbeater.cc:347] Connected to a master server at 127.20.210.62:40877
I20240501 19:57:19.137658 4004 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:19.138058 4004 heartbeater.cc:511] Master 127.20.210.62:40877 requested a full tablet report, sending...
I20240501 19:57:19.171303 3415 ts_manager.cc:194] Registered new tserver with Master: 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:19.171973 21320 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240501 19:57:19.172104 3415 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.4:33041
I20240501 19:57:19.180511 21320 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240501 19:57:19.180575 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 3608
I20240501 19:57:19.186192 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 3742
I20240501 19:57:19.671610 3605 heartbeater.cc:503] Master 127.20.210.62:40877 was elected leader, sending a full tablet report...
I20240501 19:57:20.174023 4004 heartbeater.cc:503] Master 127.20.210.62:40877 was elected leader, sending a full tablet report...
I20240501 19:57:22.943013 3415 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59428:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 19:57:22.945966 3415 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240501 19:57:22.960186 3937 tablet_service.cc:1467] Processing CreateTablet for tablet 4cb6b3485dc345128d46837f72faa202 (DEFAULT_TABLE table=rebalance_test_table_0 [id=82ab925b1e4341748da3706c139a7474]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:22.960186 3536 tablet_service.cc:1467] Processing CreateTablet for tablet 4357ef00fc0841b2a12b214feb24f1a3 (DEFAULT_TABLE table=rebalance_test_table_0 [id=82ab925b1e4341748da3706c139a7474]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:22.960629 3938 tablet_service.cc:1467] Processing CreateTablet for tablet 2ae6a0ab23a64097be80084877e0f48b (DEFAULT_TABLE table=rebalance_test_table_0 [id=82ab925b1e4341748da3706c139a7474]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:22.960189 3936 tablet_service.cc:1467] Processing CreateTablet for tablet 4357ef00fc0841b2a12b214feb24f1a3 (DEFAULT_TABLE table=rebalance_test_table_0 [id=82ab925b1e4341748da3706c139a7474]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:22.961031 3536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4357ef00fc0841b2a12b214feb24f1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:22.961031 3937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cb6b3485dc345128d46837f72faa202. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:22.961047 3538 tablet_service.cc:1467] Processing CreateTablet for tablet 2ae6a0ab23a64097be80084877e0f48b (DEFAULT_TABLE table=rebalance_test_table_0 [id=82ab925b1e4341748da3706c139a7474]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:22.961392 3538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ae6a0ab23a64097be80084877e0f48b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:22.961444 3938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ae6a0ab23a64097be80084877e0f48b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:22.960189 3537 tablet_service.cc:1467] Processing CreateTablet for tablet 4cb6b3485dc345128d46837f72faa202 (DEFAULT_TABLE table=rebalance_test_table_0 [id=82ab925b1e4341748da3706c139a7474]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:22.962297 3936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4357ef00fc0841b2a12b214feb24f1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:22.962399 3537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cb6b3485dc345128d46837f72faa202. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:22.966676 4022 tablet_bootstrap.cc:492] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086: Bootstrap starting.
I20240501 19:57:22.967742 4021 tablet_bootstrap.cc:492] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5: Bootstrap starting.
I20240501 19:57:22.968291 4022 tablet_bootstrap.cc:654] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:22.968909 4022 log.cc:824] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:22.969308 4021 tablet_bootstrap.cc:654] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:22.970196 4021 log.cc:824] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:22.970404 4022 tablet_bootstrap.cc:492] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086: No bootstrap required, opened a new log
I20240501 19:57:22.970557 4022 ts_tablet_manager.cc:1367] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240501 19:57:22.971422 4021 tablet_bootstrap.cc:492] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5: No bootstrap required, opened a new log
I20240501 19:57:22.971534 4021 ts_tablet_manager.cc:1367] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240501 19:57:22.972831 4022 raft_consensus.cc:348] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:22.973004 4022 raft_consensus.cc:374] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:22.973068 4022 raft_consensus.cc:729] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Initialized, Role: FOLLOWER
I20240501 19:57:22.973413 4022 consensus_queue.cc:260] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:22.973575 4021 raft_consensus.cc:348] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:22.973732 4021 raft_consensus.cc:374] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:22.973800 4021 raft_consensus.cc:729] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5d277450784129ab6e7b31597a48d5, State: Initialized, Role: FOLLOWER
I20240501 19:57:22.974123 4022 ts_tablet_manager.cc:1398] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20240501 19:57:22.974128 4021 consensus_queue.cc:260] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:22.974328 4022 tablet_bootstrap.cc:492] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086: Bootstrap starting.
I20240501 19:57:22.975064 4022 tablet_bootstrap.cc:654] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:22.975577 4021 ts_tablet_manager.cc:1398] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20240501 19:57:22.975800 4021 tablet_bootstrap.cc:492] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5: Bootstrap starting.
I20240501 19:57:22.976112 4022 tablet_bootstrap.cc:492] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086: No bootstrap required, opened a new log
I20240501 19:57:22.976173 4022 ts_tablet_manager.cc:1367] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:22.976397 4021 tablet_bootstrap.cc:654] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:22.976636 4022 raft_consensus.cc:348] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:22.976725 4022 raft_consensus.cc:374] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:22.976753 4022 raft_consensus.cc:729] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Initialized, Role: FOLLOWER
I20240501 19:57:22.976866 4022 consensus_queue.cc:260] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:22.977125 4022 ts_tablet_manager.cc:1398] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:22.977263 4021 tablet_bootstrap.cc:492] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5: No bootstrap required, opened a new log
I20240501 19:57:22.977257 4022 tablet_bootstrap.cc:492] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086: Bootstrap starting.
I20240501 19:57:22.977336 4021 ts_tablet_manager.cc:1367] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240501 19:57:22.977723 4021 raft_consensus.cc:348] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:22.977807 4021 raft_consensus.cc:374] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:22.977840 4021 raft_consensus.cc:729] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5d277450784129ab6e7b31597a48d5, State: Initialized, Role: FOLLOWER
I20240501 19:57:22.977947 4021 consensus_queue.cc:260] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:22.978029 4022 tablet_bootstrap.cc:654] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:22.978168 4021 ts_tablet_manager.cc:1398] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:22.978286 4021 tablet_bootstrap.cc:492] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5: Bootstrap starting.
I20240501 19:57:22.979101 4021 tablet_bootstrap.cc:654] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:22.979470 4022 tablet_bootstrap.cc:492] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086: No bootstrap required, opened a new log
I20240501 19:57:22.979537 4022 ts_tablet_manager.cc:1367] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:22.979939 4022 raft_consensus.cc:348] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:22.980070 4022 raft_consensus.cc:374] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:22.980110 4022 raft_consensus.cc:729] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Initialized, Role: FOLLOWER
I20240501 19:57:22.980134 4021 tablet_bootstrap.cc:492] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5: No bootstrap required, opened a new log
I20240501 19:57:22.980198 4021 ts_tablet_manager.cc:1367] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:22.980247 4022 consensus_queue.cc:260] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:22.980497 4021 raft_consensus.cc:348] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:22.980561 4021 raft_consensus.cc:374] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:22.980577 4021 raft_consensus.cc:729] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5d277450784129ab6e7b31597a48d5, State: Initialized, Role: FOLLOWER
I20240501 19:57:22.980679 4021 consensus_queue.cc:260] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:22.980775 4022 ts_tablet_manager.cc:1398] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:22.980918 4021 ts_tablet_manager.cc:1398] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:23.004483 4026 raft_consensus.cc:482] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:23.004602 4026 raft_consensus.cc:504] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.005353 4026 leader_election.cc:288] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143)
I20240501 19:57:23.008536 3559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4357ef00fc0841b2a12b214feb24f1a3" candidate_uuid: "1d5d277450784129ab6e7b31597a48d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af80abcc7a194cc582c7c45a62e49086" is_pre_election: true
I20240501 19:57:23.008742 3559 raft_consensus.cc:2454] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d5d277450784129ab6e7b31597a48d5 in term 0.
I20240501 19:57:23.009125 3892 leader_election.cc:302] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:23.009343 4026 raft_consensus.cc:2781] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:23.009406 4026 raft_consensus.cc:482] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:23.009435 4026 raft_consensus.cc:3037] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:23.010131 4026 raft_consensus.cc:504] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.010377 4026 leader_election.cc:288] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 1 election: Requested vote from peers af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143)
I20240501 19:57:23.010644 3559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4357ef00fc0841b2a12b214feb24f1a3" candidate_uuid: "1d5d277450784129ab6e7b31597a48d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af80abcc7a194cc582c7c45a62e49086"
I20240501 19:57:23.010733 3559 raft_consensus.cc:3037] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:23.011716 3559 raft_consensus.cc:2454] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d5d277450784129ab6e7b31597a48d5 in term 1.
I20240501 19:57:23.012001 3892 leader_election.cc:302] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:23.012168 4026 raft_consensus.cc:2781] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:23.012322 4026 raft_consensus.cc:686] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 1 LEADER]: Becoming Leader. State: Replica: 1d5d277450784129ab6e7b31597a48d5, State: Running, Role: LEADER
I20240501 19:57:23.012554 4026 consensus_queue.cc:237] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.014693 3414 catalog_manager.cc:5484] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4). New cstate: current_term: 1 leader_uuid: "1d5d277450784129ab6e7b31597a48d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:23.071318 4025 raft_consensus.cc:482] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:23.071437 4025 raft_consensus.cc:504] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.072432 4025 leader_election.cc:288] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:23.076133 3958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cb6b3485dc345128d46837f72faa202" candidate_uuid: "af80abcc7a194cc582c7c45a62e49086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d277450784129ab6e7b31597a48d5" is_pre_election: true
I20240501 19:57:23.076297 3958 raft_consensus.cc:2454] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af80abcc7a194cc582c7c45a62e49086 in term 0.
I20240501 19:57:23.076666 3494 leader_election.cc:302] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:23.076910 4025 raft_consensus.cc:2781] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:23.076967 4025 raft_consensus.cc:482] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:23.077006 4025 raft_consensus.cc:3037] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:23.077689 4025 raft_consensus.cc:504] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.077904 4025 leader_election.cc:288] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 election: Requested vote from peers 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:23.078199 3958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cb6b3485dc345128d46837f72faa202" candidate_uuid: "af80abcc7a194cc582c7c45a62e49086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d277450784129ab6e7b31597a48d5"
I20240501 19:57:23.078320 3958 raft_consensus.cc:3037] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:23.079114 3958 raft_consensus.cc:2454] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af80abcc7a194cc582c7c45a62e49086 in term 1.
I20240501 19:57:23.079358 3494 leader_election.cc:302] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:23.079535 4025 raft_consensus.cc:2781] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:23.079696 4025 raft_consensus.cc:686] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Becoming Leader. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Running, Role: LEADER
I20240501 19:57:23.080024 4025 consensus_queue.cc:237] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.082870 3414 catalog_manager.cc:5484] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 reported cstate change: term changed from 0 to 1, leader changed from <none> to af80abcc7a194cc582c7c45a62e49086 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } }
W20240501 19:57:23.127051 3606 tablet.cc:2369] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 19:57:23.133366 4005 tablet.cc:2369] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:57:23.214833 4025 raft_consensus.cc:482] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:23.214942 4025 raft_consensus.cc:504] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.215258 4025 leader_election.cc:288] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:23.215675 3958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ae6a0ab23a64097be80084877e0f48b" candidate_uuid: "af80abcc7a194cc582c7c45a62e49086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d277450784129ab6e7b31597a48d5" is_pre_election: true
I20240501 19:57:23.215821 3958 raft_consensus.cc:2454] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af80abcc7a194cc582c7c45a62e49086 in term 0.
I20240501 19:57:23.216078 3494 leader_election.cc:302] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:23.216249 4025 raft_consensus.cc:2781] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:23.216315 4025 raft_consensus.cc:482] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:23.216341 4025 raft_consensus.cc:3037] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:23.217062 4025 raft_consensus.cc:504] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.217309 4025 leader_election.cc:288] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 election: Requested vote from peers 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:23.217634 3958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ae6a0ab23a64097be80084877e0f48b" candidate_uuid: "af80abcc7a194cc582c7c45a62e49086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d277450784129ab6e7b31597a48d5"
I20240501 19:57:23.217733 3958 raft_consensus.cc:3037] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:23.218453 3958 raft_consensus.cc:2454] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af80abcc7a194cc582c7c45a62e49086 in term 1.
I20240501 19:57:23.218694 3494 leader_election.cc:302] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:23.218883 4025 raft_consensus.cc:2781] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:23.218972 4025 raft_consensus.cc:686] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Becoming Leader. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Running, Role: LEADER
I20240501 19:57:23.219107 4025 consensus_queue.cc:237] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.220494 3414 catalog_manager.cc:5484] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 reported cstate change: term changed from 0 to 1, leader changed from <none> to af80abcc7a194cc582c7c45a62e49086 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } }
W20240501 19:57:23.333483 4038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:23.333684 4038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:23.336459 4038 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240501 19:57:23.402475 3958 raft_consensus.cc:1261] T 2ae6a0ab23a64097be80084877e0f48b P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:23.403057 4025 consensus_queue.cc:1035] T 2ae6a0ab23a64097be80084877e0f48b P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:23.404310 3559 raft_consensus.cc:1261] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:23.404879 4026 consensus_queue.cc:1035] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:23.406860 3957 raft_consensus.cc:1261] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:23.407203 4025 consensus_queue.cc:1035] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:23.411134 4074 mvcc.cc:205] Tried to move back new op lower bound from 7022974744175722496 to 7022974742580957184. Current Snapshot: MvccSnapshot[applied={T|T < 7022974744175722496}]
I20240501 19:57:23.412168 4069 mvcc.cc:205] Tried to move back new op lower bound from 7022974744170160128 to 7022974743426445312. Current Snapshot: MvccSnapshot[applied={T|T < 7022974744170160128}]
INSERT report
rows total: 2000
time total: 77.4616 ms
time per row: 0.0387308 ms
I20240501 19:57:23.446955 3415 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59428:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 19:57:23.447342 3415 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_1 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240501 19:57:23.451483 3937 tablet_service.cc:1467] Processing CreateTablet for tablet e47028a7fe35419b85df1e7e5791a9ff (DEFAULT_TABLE table=rebalance_test_table_1 [id=51c847e5c2bd4a7c8e47906ec8359881]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:23.451524 3938 tablet_service.cc:1467] Processing CreateTablet for tablet addbafe6232c4790be3235342b47bf7d (DEFAULT_TABLE table=rebalance_test_table_1 [id=51c847e5c2bd4a7c8e47906ec8359881]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:23.451723 3937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e47028a7fe35419b85df1e7e5791a9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:23.451761 3936 tablet_service.cc:1467] Processing CreateTablet for tablet 931e609c374a46658347e2da5dcc0f9d (DEFAULT_TABLE table=rebalance_test_table_1 [id=51c847e5c2bd4a7c8e47906ec8359881]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:23.451956 3936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 931e609c374a46658347e2da5dcc0f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:23.451930 3537 tablet_service.cc:1467] Processing CreateTablet for tablet addbafe6232c4790be3235342b47bf7d (DEFAULT_TABLE table=rebalance_test_table_1 [id=51c847e5c2bd4a7c8e47906ec8359881]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:23.452129 3537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addbafe6232c4790be3235342b47bf7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:23.453440 4021 tablet_bootstrap.cc:492] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5: Bootstrap starting.
I20240501 19:57:23.453928 3536 tablet_service.cc:1467] Processing CreateTablet for tablet e47028a7fe35419b85df1e7e5791a9ff (DEFAULT_TABLE table=rebalance_test_table_1 [id=51c847e5c2bd4a7c8e47906ec8359881]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:23.454144 3536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e47028a7fe35419b85df1e7e5791a9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:23.454255 4021 tablet_bootstrap.cc:654] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:23.454352 4022 tablet_bootstrap.cc:492] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086: Bootstrap starting.
I20240501 19:57:23.454767 3938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addbafe6232c4790be3235342b47bf7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:23.455089 4022 tablet_bootstrap.cc:654] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:23.454142 3538 tablet_service.cc:1467] Processing CreateTablet for tablet 931e609c374a46658347e2da5dcc0f9d (DEFAULT_TABLE table=rebalance_test_table_1 [id=51c847e5c2bd4a7c8e47906ec8359881]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:23.455842 3538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 931e609c374a46658347e2da5dcc0f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:23.456895 4021 tablet_bootstrap.cc:492] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5: No bootstrap required, opened a new log
I20240501 19:57:23.456970 4021 ts_tablet_manager.cc:1367] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:57:23.457306 4021 raft_consensus.cc:348] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.457410 4021 raft_consensus.cc:374] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:23.457443 4021 raft_consensus.cc:729] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5d277450784129ab6e7b31597a48d5, State: Initialized, Role: FOLLOWER
I20240501 19:57:23.457569 4021 consensus_queue.cc:260] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.457736 4022 tablet_bootstrap.cc:492] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086: No bootstrap required, opened a new log
I20240501 19:57:23.457821 4022 ts_tablet_manager.cc:1367] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:57:23.457841 4021 ts_tablet_manager.cc:1398] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:23.457980 4021 tablet_bootstrap.cc:492] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5: Bootstrap starting.
I20240501 19:57:23.458218 4022 raft_consensus.cc:348] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.458350 4022 raft_consensus.cc:374] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:23.458386 4022 raft_consensus.cc:729] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Initialized, Role: FOLLOWER
I20240501 19:57:23.458499 4022 consensus_queue.cc:260] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.458717 4022 ts_tablet_manager.cc:1398] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:23.458766 4021 tablet_bootstrap.cc:654] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:23.458892 4022 tablet_bootstrap.cc:492] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086: Bootstrap starting.
I20240501 19:57:23.459756 4022 tablet_bootstrap.cc:654] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:23.460070 4021 tablet_bootstrap.cc:492] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5: No bootstrap required, opened a new log
I20240501 19:57:23.460136 4021 ts_tablet_manager.cc:1367] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:23.460491 4021 raft_consensus.cc:348] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.460584 4021 raft_consensus.cc:374] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:23.460613 4021 raft_consensus.cc:729] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5d277450784129ab6e7b31597a48d5, State: Initialized, Role: FOLLOWER
I20240501 19:57:23.460731 4021 consensus_queue.cc:260] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.460816 4022 tablet_bootstrap.cc:492] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086: No bootstrap required, opened a new log
I20240501 19:57:23.460870 4022 ts_tablet_manager.cc:1367] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:23.460978 4021 ts_tablet_manager.cc:1398] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:23.461095 4021 tablet_bootstrap.cc:492] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5: Bootstrap starting.
I20240501 19:57:23.461205 4022 raft_consensus.cc:348] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.461272 4022 raft_consensus.cc:374] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:23.461297 4022 raft_consensus.cc:729] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Initialized, Role: FOLLOWER
I20240501 19:57:23.461407 4022 consensus_queue.cc:260] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.461630 4022 ts_tablet_manager.cc:1398] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:23.461774 4022 tablet_bootstrap.cc:492] T 931e609c374a46658347e2da5dcc0f9d P af80abcc7a194cc582c7c45a62e49086: Bootstrap starting.
I20240501 19:57:23.461797 4021 tablet_bootstrap.cc:654] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:23.462705 4022 tablet_bootstrap.cc:654] T 931e609c374a46658347e2da5dcc0f9d P af80abcc7a194cc582c7c45a62e49086: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:23.463441 4021 tablet_bootstrap.cc:492] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5: No bootstrap required, opened a new log
I20240501 19:57:23.463495 4021 ts_tablet_manager.cc:1367] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:23.463773 4022 tablet_bootstrap.cc:492] T 931e609c374a46658347e2da5dcc0f9d P af80abcc7a194cc582c7c45a62e49086: No bootstrap required, opened a new log
I20240501 19:57:23.463775 4021 raft_consensus.cc:348] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.463825 4022 ts_tablet_manager.cc:1367] T 931e609c374a46658347e2da5dcc0f9d P af80abcc7a194cc582c7c45a62e49086: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:23.463837 4021 raft_consensus.cc:374] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:23.463856 4021 raft_consensus.cc:729] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5d277450784129ab6e7b31597a48d5, State: Initialized, Role: FOLLOWER
I20240501 19:57:23.463938 4021 consensus_queue.cc:260] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.464143 4021 ts_tablet_manager.cc:1398] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:23.464239 4022 raft_consensus.cc:348] T 931e609c374a46658347e2da5dcc0f9d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.464318 4022 raft_consensus.cc:374] T 931e609c374a46658347e2da5dcc0f9d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:23.464363 4022 raft_consensus.cc:729] T 931e609c374a46658347e2da5dcc0f9d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Initialized, Role: FOLLOWER
I20240501 19:57:23.464466 4022 consensus_queue.cc:260] T 931e609c374a46658347e2da5dcc0f9d P af80abcc7a194cc582c7c45a62e49086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.464677 4022 ts_tablet_manager.cc:1398] T 931e609c374a46658347e2da5dcc0f9d P af80abcc7a194cc582c7c45a62e49086: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:23.545984 4035 raft_consensus.cc:482] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:23.546099 4035 raft_consensus.cc:504] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.546409 4035 leader_election.cc:288] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:23.546768 3957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47028a7fe35419b85df1e7e5791a9ff" candidate_uuid: "af80abcc7a194cc582c7c45a62e49086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d277450784129ab6e7b31597a48d5" is_pre_election: true
I20240501 19:57:23.546880 3957 raft_consensus.cc:2454] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af80abcc7a194cc582c7c45a62e49086 in term 0.
I20240501 19:57:23.547137 3494 leader_election.cc:302] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:23.547286 4035 raft_consensus.cc:2781] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:23.547338 4035 raft_consensus.cc:482] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:23.547386 4035 raft_consensus.cc:3037] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:23.548121 4035 raft_consensus.cc:504] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.548347 4035 leader_election.cc:288] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 election: Requested vote from peers 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:23.548666 3957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47028a7fe35419b85df1e7e5791a9ff" candidate_uuid: "af80abcc7a194cc582c7c45a62e49086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d277450784129ab6e7b31597a48d5"
I20240501 19:57:23.548803 3957 raft_consensus.cc:3037] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:23.549579 3957 raft_consensus.cc:2454] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af80abcc7a194cc582c7c45a62e49086 in term 1.
I20240501 19:57:23.549860 3494 leader_election.cc:302] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:23.550034 4035 raft_consensus.cc:2781] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:23.550118 4035 raft_consensus.cc:686] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Becoming Leader. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Running, Role: LEADER
I20240501 19:57:23.550282 4035 consensus_queue.cc:237] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.551482 3413 catalog_manager.cc:5484] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 reported cstate change: term changed from 0 to 1, leader changed from <none> to af80abcc7a194cc582c7c45a62e49086 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:23.561307 4081 raft_consensus.cc:482] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:23.561378 4081 raft_consensus.cc:504] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.561614 4081 leader_election.cc:288] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143)
I20240501 19:57:23.561897 3559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addbafe6232c4790be3235342b47bf7d" candidate_uuid: "1d5d277450784129ab6e7b31597a48d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af80abcc7a194cc582c7c45a62e49086" is_pre_election: true
I20240501 19:57:23.561980 3559 raft_consensus.cc:2454] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d5d277450784129ab6e7b31597a48d5 in term 0.
I20240501 19:57:23.562188 3892 leader_election.cc:302] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:23.562348 4081 raft_consensus.cc:2781] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:23.562404 4081 raft_consensus.cc:482] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:23.562430 4081 raft_consensus.cc:3037] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:23.562991 4081 raft_consensus.cc:504] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.563184 4081 leader_election.cc:288] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 1 election: Requested vote from peers af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143)
I20240501 19:57:23.563444 3559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "addbafe6232c4790be3235342b47bf7d" candidate_uuid: "1d5d277450784129ab6e7b31597a48d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af80abcc7a194cc582c7c45a62e49086"
I20240501 19:57:23.563521 3559 raft_consensus.cc:3037] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:23.564148 3559 raft_consensus.cc:2454] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d5d277450784129ab6e7b31597a48d5 in term 1.
I20240501 19:57:23.564416 3892 leader_election.cc:302] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:23.564546 4081 raft_consensus.cc:2781] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:23.564621 4081 raft_consensus.cc:686] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 1 LEADER]: Becoming Leader. State: Replica: 1d5d277450784129ab6e7b31597a48d5, State: Running, Role: LEADER
I20240501 19:57:23.564713 4081 consensus_queue.cc:237] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.565809 3413 catalog_manager.cc:5484] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4). New cstate: current_term: 1 leader_uuid: "1d5d277450784129ab6e7b31597a48d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:23.796100 4081 raft_consensus.cc:482] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:23.796238 4081 raft_consensus.cc:504] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.796677 4081 leader_election.cc:288] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143)
I20240501 19:57:23.797128 3559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "931e609c374a46658347e2da5dcc0f9d" candidate_uuid: "1d5d277450784129ab6e7b31597a48d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af80abcc7a194cc582c7c45a62e49086" is_pre_election: true
I20240501 19:57:23.797257 3559 raft_consensus.cc:2454] T 931e609c374a46658347e2da5dcc0f9d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d5d277450784129ab6e7b31597a48d5 in term 0.
I20240501 19:57:23.797528 3892 leader_election.cc:302] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:23.797699 4081 raft_consensus.cc:2781] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:23.797765 4081 raft_consensus.cc:482] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:23.797804 4081 raft_consensus.cc:3037] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:23.798564 4081 raft_consensus.cc:504] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.798815 4081 leader_election.cc:288] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 1 election: Requested vote from peers af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143)
I20240501 19:57:23.799176 3559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "931e609c374a46658347e2da5dcc0f9d" candidate_uuid: "1d5d277450784129ab6e7b31597a48d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af80abcc7a194cc582c7c45a62e49086"
I20240501 19:57:23.799302 3559 raft_consensus.cc:3037] T 931e609c374a46658347e2da5dcc0f9d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:23.799988 3559 raft_consensus.cc:2454] T 931e609c374a46658347e2da5dcc0f9d P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d5d277450784129ab6e7b31597a48d5 in term 1.
I20240501 19:57:23.800227 3892 leader_election.cc:302] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:23.800370 4081 raft_consensus.cc:2781] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:23.800453 4081 raft_consensus.cc:686] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [term 1 LEADER]: Becoming Leader. State: Replica: 1d5d277450784129ab6e7b31597a48d5, State: Running, Role: LEADER
I20240501 19:57:23.800588 4081 consensus_queue.cc:237] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:23.801949 3413 catalog_manager.cc:5484] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4). New cstate: current_term: 1 leader_uuid: "1d5d277450784129ab6e7b31597a48d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: UNKNOWN } } }
W20240501 19:57:23.905736 4094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:23.905937 4094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:23.908679 4094 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240501 19:57:23.974522 3957 raft_consensus.cc:1261] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:23.974905 4035 consensus_queue.cc:1035] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:23.977628 3559 raft_consensus.cc:1261] T 931e609c374a46658347e2da5dcc0f9d P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:23.978039 4081 consensus_queue.cc:1035] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:23.978734 3559 raft_consensus.cc:1261] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:23.979087 4081 consensus_queue.cc:1035] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 75.5228 ms
time per row: 0.0377614 ms
I20240501 19:57:24.020844 3408 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59428:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 19:57:24.021216 3408 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_2 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240501 19:57:24.025415 3538 tablet_service.cc:1467] Processing CreateTablet for tablet cb4bd5cf721f43838ca02bfdb3797310 (DEFAULT_TABLE table=rebalance_test_table_2 [id=adfec7d1f5ff4471ad6da4c37fffa367]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:24.025422 3938 tablet_service.cc:1467] Processing CreateTablet for tablet cb4bd5cf721f43838ca02bfdb3797310 (DEFAULT_TABLE table=rebalance_test_table_2 [id=adfec7d1f5ff4471ad6da4c37fffa367]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:24.025676 3538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4bd5cf721f43838ca02bfdb3797310. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:24.025679 3938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4bd5cf721f43838ca02bfdb3797310. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:24.025856 3536 tablet_service.cc:1467] Processing CreateTablet for tablet 78f3dca63e9b419ca0f55261bdd7875d (DEFAULT_TABLE table=rebalance_test_table_2 [id=adfec7d1f5ff4471ad6da4c37fffa367]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:24.026057 3536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78f3dca63e9b419ca0f55261bdd7875d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:24.027340 4144 tablet_bootstrap.cc:492] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086: Bootstrap starting.
I20240501 19:57:24.027475 3537 tablet_service.cc:1467] Processing CreateTablet for tablet 779aebb72b814aa4b3818a6d93f5ffbf (DEFAULT_TABLE table=rebalance_test_table_2 [id=adfec7d1f5ff4471ad6da4c37fffa367]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:24.027658 3537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779aebb72b814aa4b3818a6d93f5ffbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:24.027812 4145 tablet_bootstrap.cc:492] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5: Bootstrap starting.
I20240501 19:57:24.028072 4144 tablet_bootstrap.cc:654] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:24.028504 4145 tablet_bootstrap.cc:654] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:24.029289 4144 tablet_bootstrap.cc:492] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086: No bootstrap required, opened a new log
I20240501 19:57:24.029363 4144 ts_tablet_manager.cc:1367] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:24.029564 3938 tablet_service.cc:1467] Processing CreateTablet for tablet 78f3dca63e9b419ca0f55261bdd7875d (DEFAULT_TABLE table=rebalance_test_table_2 [id=adfec7d1f5ff4471ad6da4c37fffa367]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:24.029577 3936 tablet_service.cc:1467] Processing CreateTablet for tablet 779aebb72b814aa4b3818a6d93f5ffbf (DEFAULT_TABLE table=rebalance_test_table_2 [id=adfec7d1f5ff4471ad6da4c37fffa367]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:24.029830 3938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78f3dca63e9b419ca0f55261bdd7875d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:24.029996 3936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 779aebb72b814aa4b3818a6d93f5ffbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:24.030179 4144 raft_consensus.cc:348] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.030332 4144 raft_consensus.cc:374] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:24.030372 4145 tablet_bootstrap.cc:492] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5: No bootstrap required, opened a new log
I20240501 19:57:24.030386 4144 raft_consensus.cc:729] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Initialized, Role: FOLLOWER
I20240501 19:57:24.030458 4145 ts_tablet_manager.cc:1367] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:24.030534 4144 consensus_queue.cc:260] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.030862 4144 ts_tablet_manager.cc:1398] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:24.031004 4144 tablet_bootstrap.cc:492] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086: Bootstrap starting.
I20240501 19:57:24.031239 4145 raft_consensus.cc:348] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.031363 4145 raft_consensus.cc:374] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:24.031406 4145 raft_consensus.cc:729] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5d277450784129ab6e7b31597a48d5, State: Initialized, Role: FOLLOWER
I20240501 19:57:24.031533 4145 consensus_queue.cc:260] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.031859 4145 ts_tablet_manager.cc:1398] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:24.032040 4145 tablet_bootstrap.cc:492] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5: Bootstrap starting.
I20240501 19:57:24.032496 4144 tablet_bootstrap.cc:654] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:24.032860 4145 tablet_bootstrap.cc:654] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:24.033778 4144 tablet_bootstrap.cc:492] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086: No bootstrap required, opened a new log
I20240501 19:57:24.033851 4144 ts_tablet_manager.cc:1367] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240501 19:57:24.034216 4144 raft_consensus.cc:348] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.034291 4145 tablet_bootstrap.cc:492] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5: No bootstrap required, opened a new log
I20240501 19:57:24.034330 4144 raft_consensus.cc:374] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:24.034368 4144 raft_consensus.cc:729] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Initialized, Role: FOLLOWER
I20240501 19:57:24.034380 4145 ts_tablet_manager.cc:1367] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:24.034461 4144 consensus_queue.cc:260] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.034667 4144 ts_tablet_manager.cc:1398] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:24.034726 4145 raft_consensus.cc:348] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.034804 4145 raft_consensus.cc:374] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:24.034828 4145 raft_consensus.cc:729] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5d277450784129ab6e7b31597a48d5, State: Initialized, Role: FOLLOWER
I20240501 19:57:24.034823 4144 tablet_bootstrap.cc:492] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086: Bootstrap starting.
I20240501 19:57:24.034915 4145 consensus_queue.cc:260] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.035123 4145 ts_tablet_manager.cc:1398] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:24.035271 4145 tablet_bootstrap.cc:492] T 779aebb72b814aa4b3818a6d93f5ffbf P 1d5d277450784129ab6e7b31597a48d5: Bootstrap starting.
I20240501 19:57:24.035564 4144 tablet_bootstrap.cc:654] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:24.035853 4145 tablet_bootstrap.cc:654] T 779aebb72b814aa4b3818a6d93f5ffbf P 1d5d277450784129ab6e7b31597a48d5: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:24.036520 4144 tablet_bootstrap.cc:492] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086: No bootstrap required, opened a new log
I20240501 19:57:24.036589 4144 ts_tablet_manager.cc:1367] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 19:57:24.036665 4145 tablet_bootstrap.cc:492] T 779aebb72b814aa4b3818a6d93f5ffbf P 1d5d277450784129ab6e7b31597a48d5: No bootstrap required, opened a new log
I20240501 19:57:24.036721 4145 ts_tablet_manager.cc:1367] T 779aebb72b814aa4b3818a6d93f5ffbf P 1d5d277450784129ab6e7b31597a48d5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:24.036926 4144 raft_consensus.cc:348] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:24.036993 4144 raft_consensus.cc:374] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:24.037019 4144 raft_consensus.cc:729] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Initialized, Role: FOLLOWER
I20240501 19:57:24.037062 4145 raft_consensus.cc:348] T 779aebb72b814aa4b3818a6d93f5ffbf P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:24.037128 4145 raft_consensus.cc:374] T 779aebb72b814aa4b3818a6d93f5ffbf P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:24.037158 4145 raft_consensus.cc:729] T 779aebb72b814aa4b3818a6d93f5ffbf P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5d277450784129ab6e7b31597a48d5, State: Initialized, Role: FOLLOWER
I20240501 19:57:24.037145 4144 consensus_queue.cc:260] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:24.037369 4144 ts_tablet_manager.cc:1398] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:24.037639 4145 consensus_queue.cc:260] T 779aebb72b814aa4b3818a6d93f5ffbf P 1d5d277450784129ab6e7b31597a48d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:24.037959 4145 ts_tablet_manager.cc:1398] T 779aebb72b814aa4b3818a6d93f5ffbf P 1d5d277450784129ab6e7b31597a48d5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:24.106020 4137 raft_consensus.cc:482] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:24.106160 4137 raft_consensus.cc:504] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.106530 4137 leader_election.cc:288] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:24.107038 3957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4bd5cf721f43838ca02bfdb3797310" candidate_uuid: "af80abcc7a194cc582c7c45a62e49086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d277450784129ab6e7b31597a48d5" is_pre_election: true
I20240501 19:57:24.107173 3957 raft_consensus.cc:2454] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af80abcc7a194cc582c7c45a62e49086 in term 0.
I20240501 19:57:24.107491 3494 leader_election.cc:302] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:24.107676 4137 raft_consensus.cc:2781] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:24.107734 4137 raft_consensus.cc:482] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:24.107776 4137 raft_consensus.cc:3037] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:24.108600 4137 raft_consensus.cc:504] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.108877 4137 leader_election.cc:288] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 election: Requested vote from peers 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:24.109234 3957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4bd5cf721f43838ca02bfdb3797310" candidate_uuid: "af80abcc7a194cc582c7c45a62e49086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d277450784129ab6e7b31597a48d5"
I20240501 19:57:24.109359 3957 raft_consensus.cc:3037] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:24.110090 3957 raft_consensus.cc:2454] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af80abcc7a194cc582c7c45a62e49086 in term 1.
I20240501 19:57:24.110389 3494 leader_election.cc:302] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:24.110556 4137 raft_consensus.cc:2781] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:24.110635 4137 raft_consensus.cc:686] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Becoming Leader. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Running, Role: LEADER
I20240501 19:57:24.110764 4137 consensus_queue.cc:237] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.112211 3414 catalog_manager.cc:5484] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 reported cstate change: term changed from 0 to 1, leader changed from <none> to af80abcc7a194cc582c7c45a62e49086 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:24.246410 4137 raft_consensus.cc:482] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:24.246552 4137 raft_consensus.cc:504] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:24.246887 4137 leader_election.cc:288] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:24.247314 3957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779aebb72b814aa4b3818a6d93f5ffbf" candidate_uuid: "af80abcc7a194cc582c7c45a62e49086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d277450784129ab6e7b31597a48d5" is_pre_election: true
I20240501 19:57:24.247435 3957 raft_consensus.cc:2454] T 779aebb72b814aa4b3818a6d93f5ffbf P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af80abcc7a194cc582c7c45a62e49086 in term 0.
I20240501 19:57:24.247725 3494 leader_election.cc:302] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:24.247902 4137 raft_consensus.cc:2781] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:24.247958 4137 raft_consensus.cc:482] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:24.247985 4137 raft_consensus.cc:3037] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:24.248725 4137 raft_consensus.cc:504] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:24.248973 4137 leader_election.cc:288] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 election: Requested vote from peers 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:24.249336 3957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "779aebb72b814aa4b3818a6d93f5ffbf" candidate_uuid: "af80abcc7a194cc582c7c45a62e49086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d277450784129ab6e7b31597a48d5"
I20240501 19:57:24.249459 3957 raft_consensus.cc:3037] T 779aebb72b814aa4b3818a6d93f5ffbf P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:24.250180 3957 raft_consensus.cc:2454] T 779aebb72b814aa4b3818a6d93f5ffbf P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af80abcc7a194cc582c7c45a62e49086 in term 1.
I20240501 19:57:24.250464 3494 leader_election.cc:302] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:24.250656 4137 raft_consensus.cc:2781] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:24.250757 4137 raft_consensus.cc:686] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Becoming Leader. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Running, Role: LEADER
I20240501 19:57:24.250895 4137 consensus_queue.cc:237] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } }
I20240501 19:57:24.252255 3414 catalog_manager.cc:5484] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 reported cstate change: term changed from 0 to 1, leader changed from <none> to af80abcc7a194cc582c7c45a62e49086 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:24.343768 4137 raft_consensus.cc:482] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:24.343909 4137 raft_consensus.cc:504] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.344255 4137 leader_election.cc:288] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:24.344802 3957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78f3dca63e9b419ca0f55261bdd7875d" candidate_uuid: "af80abcc7a194cc582c7c45a62e49086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d277450784129ab6e7b31597a48d5" is_pre_election: true
I20240501 19:57:24.344947 3957 raft_consensus.cc:2454] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af80abcc7a194cc582c7c45a62e49086 in term 0.
I20240501 19:57:24.345245 3494 leader_election.cc:302] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:24.345424 4137 raft_consensus.cc:2781] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:24.345490 4137 raft_consensus.cc:482] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:24.345522 4137 raft_consensus.cc:3037] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:24.346273 4137 raft_consensus.cc:504] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.346514 4137 leader_election.cc:288] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 election: Requested vote from peers 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:24.346851 3957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78f3dca63e9b419ca0f55261bdd7875d" candidate_uuid: "af80abcc7a194cc582c7c45a62e49086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d277450784129ab6e7b31597a48d5"
I20240501 19:57:24.346966 3957 raft_consensus.cc:3037] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:24.347606 3957 raft_consensus.cc:2454] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af80abcc7a194cc582c7c45a62e49086 in term 1.
I20240501 19:57:24.347887 3494 leader_election.cc:302] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, af80abcc7a194cc582c7c45a62e49086; no voters:
I20240501 19:57:24.348023 4137 raft_consensus.cc:2781] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:24.348099 4137 raft_consensus.cc:686] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Becoming Leader. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Running, Role: LEADER
I20240501 19:57:24.348232 4137 consensus_queue.cc:237] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } }
I20240501 19:57:24.349880 3414 catalog_manager.cc:5484] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 reported cstate change: term changed from 0 to 1, leader changed from <none> to af80abcc7a194cc582c7c45a62e49086 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: UNKNOWN } } }
W20240501 19:57:24.482344 4155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:24.482519 4155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:24.485380 4155 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240501 19:57:24.550176 3957 raft_consensus.cc:1261] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:24.550812 4035 consensus_queue.cc:1035] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:24.551483 3958 raft_consensus.cc:1261] T 779aebb72b814aa4b3818a6d93f5ffbf P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:24.551944 3957 raft_consensus.cc:1261] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:24.552425 4137 consensus_queue.cc:1035] T 779aebb72b814aa4b3818a6d93f5ffbf P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:24.552673 4137 consensus_queue.cc:1035] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 70.7688 ms
time per row: 0.0353844 ms
I20240501 19:57:24.591704 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.2:34623
--local_ip_for_outbound_sockets=127.20.210.2
--tserver_master_addrs=127.20.210.62:40877
--webserver_port=36663
--webserver_interface=127.20.210.2
--builtin_ntp_servers=127.20.210.20:34645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 19:57:24.692615 4201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:24.692781 4201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:24.692857 4201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:24.695576 4201 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:24.695628 4201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:24.695711 4201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.2
I20240501 19:57:24.698657 4201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:34645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.2:34623
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.20.210.2
--webserver_port=36663
--tserver_master_addrs=127.20.210.62:40877
--never_fsync=true
--heap_profile_path=/tmp/kudu.4201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.2
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:24.699342 4201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:24.700299 4201 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:24.707940 4206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:24.708413 4207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:24.709134 4209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:24.709893 4201 server_base.cc:1034] running on GCE node
I20240501 19:57:24.710248 4201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:24.710697 4201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:24.711833 4201 hybrid_clock.cc:648] HybridClock initialized: now 1714593444711813 us; error 37 us; skew 500 ppm
I20240501 19:57:24.713284 4201 webserver.cc:460] Webserver started at http://127.20.210.2:36663/ using document root <none> and password file <none>
I20240501 19:57:24.713716 4201 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:24.713795 4201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:24.716589 4201 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:24.718166 4215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:24.719066 4201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:24.719167 4201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "c1a344780dbb4de9bcb624ac5d91d469"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:24.719512 4201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:24.730639 4201 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:24.731071 4201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:24.731204 4201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:24.731606 4201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:24.732301 4201 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:24.732373 4201 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:24.732421 4201 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:24.732463 4201 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:24.740252 4201 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.2:34623
I20240501 19:57:24.740269 4329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.2:34623 every 8 connection(s)
I20240501 19:57:24.741075 4201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 19:57:24.746348 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 4201
I20240501 19:57:24.746788 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.3:38243
--local_ip_for_outbound_sockets=127.20.210.3
--tserver_master_addrs=127.20.210.62:40877
--webserver_port=41613
--webserver_interface=127.20.210.3
--builtin_ntp_servers=127.20.210.20:34645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:24.748589 4330 heartbeater.cc:347] Connected to a master server at 127.20.210.62:40877
I20240501 19:57:24.748780 4330 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:24.749231 4330 heartbeater.cc:511] Master 127.20.210.62:40877 requested a full tablet report, sending...
I20240501 19:57:24.750229 3414 ts_manager.cc:194] Re-registered known tserver with Master: c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2:34623)
I20240501 19:57:24.750798 3414 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.2:37823
W20240501 19:57:24.839560 4333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:24.839735 4333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:24.839782 4333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:24.842460 4333 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:24.842509 4333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:24.842581 4333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.3
I20240501 19:57:24.845695 4333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:34645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.3:38243
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.20.210.3
--webserver_port=41613
--tserver_master_addrs=127.20.210.62:40877
--never_fsync=true
--heap_profile_path=/tmp/kudu.4333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.3
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:24.846345 4333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:24.847186 4333 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:24.854338 4338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:24.854853 4339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:24.855639 4341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:24.856431 4333 server_base.cc:1034] running on GCE node
I20240501 19:57:24.856775 4333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:24.857230 4333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:24.858372 4333 hybrid_clock.cc:648] HybridClock initialized: now 1714593444858349 us; error 36 us; skew 500 ppm
I20240501 19:57:24.859997 4333 webserver.cc:460] Webserver started at http://127.20.210.3:41613/ using document root <none> and password file <none>
I20240501 19:57:24.860495 4333 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:24.860564 4333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:24.863323 4333 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 19:57:24.865046 4347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:24.865900 4333 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:24.865999 4333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "f0f5056255ed4763b222e74191d4d582"
format_stamp: "Formatted at 2024-05-01 19:57:18 on dist-test-slave-vgh4"
I20240501 19:57:24.866381 4333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:24.888238 4333 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:24.888671 4333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:24.888803 4333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:24.889272 4333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:24.890071 4333 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:24.890142 4333 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:24.890194 4333 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:24.890273 4333 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:24.897720 4333 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.3:38243
I20240501 19:57:24.897743 4461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.3:38243 every 8 connection(s)
I20240501 19:57:24.898561 4333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 19:57:24.902037 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 4333
I20240501 19:57:24.906296 4462 heartbeater.cc:347] Connected to a master server at 127.20.210.62:40877
I20240501 19:57:24.906463 4462 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:24.906847 4462 heartbeater.cc:511] Master 127.20.210.62:40877 requested a full tablet report, sending...
I20240501 19:57:24.907820 3414 ts_manager.cc:194] Re-registered known tserver with Master: f0f5056255ed4763b222e74191d4d582 (127.20.210.3:38243)
I20240501 19:57:24.908547 3414 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.3:52531
I20240501 19:57:25.064550 4263 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:25.064754 3537 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:25.065132 3936 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:25.067075 4394 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:25.143087 3936 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:25.143658 3537 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:25.147120 4263 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:25.156890 4394 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:25.248829 3537 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:25.249204 4263 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:25.249650 3936 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:25.250324 4394 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:25.324862 3936 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:25.325387 3537 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:25.325635 4263 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:25.325875 4394 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:25.359130 3956 consensus_queue.cc:237] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } }
I20240501 19:57:25.360451 3558 raft_consensus.cc:1261] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:25.360831 4134 consensus_queue.cc:1035] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:25.362591 4189 raft_consensus.cc:2932] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } } }
I20240501 19:57:25.362962 3558 raft_consensus.cc:2932] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } } }
I20240501 19:57:25.365005 3414 catalog_manager.cc:5484] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 reported cstate change: config changed from index -1 to 4, NON_VOTER c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) added. New cstate: current_term: 1 leader_uuid: "1d5d277450784129ab6e7b31597a48d5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 19:57:25.365172 3893 consensus_peers.cc:485] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 -> Peer c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2:34623): Couldn't send request to peer c1a344780dbb4de9bcb624ac5d91d469. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4357ef00fc0841b2a12b214feb24f1a3. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:25.371644 3558 consensus_queue.cc:237] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } }
I20240501 19:57:25.373016 3956 raft_consensus.cc:1261] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:25.373344 4035 consensus_queue.cc:1035] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:25.374837 4616 raft_consensus.cc:2932] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f0f5056255ed4763b222e74191d4d582 (127.20.210.3) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } } }
I20240501 19:57:25.375154 3956 raft_consensus.cc:2932] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f0f5056255ed4763b222e74191d4d582 (127.20.210.3) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } } }
I20240501 19:57:25.376789 3408 catalog_manager.cc:5484] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 reported cstate change: config changed from index -1 to 4, NON_VOTER f0f5056255ed4763b222e74191d4d582 (127.20.210.3) added. New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 19:57:25.377696 3494 consensus_peers.cc:485] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 -> Peer f0f5056255ed4763b222e74191d4d582 (127.20.210.3:38243): Couldn't send request to peer f0f5056255ed4763b222e74191d4d582. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 78f3dca63e9b419ca0f55261bdd7875d. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:25.383723 3956 consensus_queue.cc:237] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } }
I20240501 19:57:25.384814 3558 raft_consensus.cc:1261] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 19:57:25.384953 3893 consensus_peers.cc:485] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 -> Peer c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2:34623): Couldn't send request to peer c1a344780dbb4de9bcb624ac5d91d469. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: addbafe6232c4790be3235342b47bf7d. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:25.385190 4134 consensus_queue.cc:1035] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:25.386538 4189 raft_consensus.cc:2932] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } } }
I20240501 19:57:25.386958 3558 raft_consensus.cc:2932] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } } }
I20240501 19:57:25.388473 3414 catalog_manager.cc:5484] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 reported cstate change: config changed from index -1 to 4, NON_VOTER c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) added. New cstate: current_term: 1 leader_uuid: "1d5d277450784129ab6e7b31597a48d5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:25.394662 3558 consensus_queue.cc:237] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } }
I20240501 19:57:25.395673 3956 raft_consensus.cc:1261] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 19:57:25.395831 3494 consensus_peers.cc:485] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 -> Peer f0f5056255ed4763b222e74191d4d582 (127.20.210.3:38243): Couldn't send request to peer f0f5056255ed4763b222e74191d4d582. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e47028a7fe35419b85df1e7e5791a9ff. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:25.395978 4035 consensus_queue.cc:1035] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:25.397266 4616 raft_consensus.cc:2932] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f0f5056255ed4763b222e74191d4d582 (127.20.210.3) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } } }
I20240501 19:57:25.397643 3956 raft_consensus.cc:2932] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f0f5056255ed4763b222e74191d4d582 (127.20.210.3) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } } }
I20240501 19:57:25.399289 3408 catalog_manager.cc:5484] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 reported cstate change: config changed from index -1 to 4, NON_VOTER f0f5056255ed4763b222e74191d4d582 (127.20.210.3) added. New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:25.404920 3558 consensus_queue.cc:237] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } }
I20240501 19:57:25.405972 3956 raft_consensus.cc:1261] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:25.406297 4633 consensus_queue.cc:1035] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:25.407581 4616 raft_consensus.cc:2932] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } } }
I20240501 19:57:25.408833 3956 raft_consensus.cc:2932] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } } }
I20240501 19:57:25.409339 3414 catalog_manager.cc:5484] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 reported cstate change: config changed from index -1 to 4, NON_VOTER c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) added. New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 19:57:25.409686 3494 consensus_peers.cc:485] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 -> Peer c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2:34623): Couldn't send request to peer c1a344780dbb4de9bcb624ac5d91d469. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cb4bd5cf721f43838ca02bfdb3797310. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:25.415742 3558 consensus_queue.cc:237] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } }
I20240501 19:57:25.416747 3956 raft_consensus.cc:1261] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240501 19:57:25.416918 3494 consensus_peers.cc:485] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 -> Peer f0f5056255ed4763b222e74191d4d582 (127.20.210.3:38243): Couldn't send request to peer f0f5056255ed4763b222e74191d4d582. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cb6b3485dc345128d46837f72faa202. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:25.417205 4035 consensus_queue.cc:1035] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:25.418673 4633 raft_consensus.cc:2932] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f0f5056255ed4763b222e74191d4d582 (127.20.210.3) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } } }
I20240501 19:57:25.419148 3956 raft_consensus.cc:2932] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f0f5056255ed4763b222e74191d4d582 (127.20.210.3) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } } }
I20240501 19:57:25.420855 3408 catalog_manager.cc:5484] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 reported cstate change: config changed from index -1 to 4, NON_VOTER f0f5056255ed4763b222e74191d4d582 (127.20.210.3) added. New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:25.752264 4330 heartbeater.cc:503] Master 127.20.210.62:40877 was elected leader, sending a full tablet report...
I20240501 19:57:25.841701 4707 ts_tablet_manager.cc:902] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469: Initiating tablet copy from peer 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:25.842468 4707 tablet_copy_client.cc:308] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:42263
I20240501 19:57:25.845727 3978 tablet_copy_service.cc:140] P 1d5d277450784129ab6e7b31597a48d5: Received BeginTabletCopySession request for tablet addbafe6232c4790be3235342b47bf7d from peer c1a344780dbb4de9bcb624ac5d91d469 ({username='slave'} at 127.20.210.2:44929)
I20240501 19:57:25.845844 3978 tablet_copy_service.cc:161] P 1d5d277450784129ab6e7b31597a48d5: Beginning new tablet copy session on tablet addbafe6232c4790be3235342b47bf7d from peer c1a344780dbb4de9bcb624ac5d91d469 at {username='slave'} at 127.20.210.2:44929: session id = c1a344780dbb4de9bcb624ac5d91d469-addbafe6232c4790be3235342b47bf7d
I20240501 19:57:25.847088 3978 tablet_copy_source_session.cc:205] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:25.847630 4713 ts_tablet_manager.cc:902] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582: Initiating tablet copy from peer af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143)
I20240501 19:57:25.848407 4713 tablet_copy_client.cc:308] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:36143
I20240501 19:57:25.850728 4707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet addbafe6232c4790be3235342b47bf7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:25.852159 3579 tablet_copy_service.cc:140] P af80abcc7a194cc582c7c45a62e49086: Received BeginTabletCopySession request for tablet 4cb6b3485dc345128d46837f72faa202 from peer f0f5056255ed4763b222e74191d4d582 ({username='slave'} at 127.20.210.3:59007)
I20240501 19:57:25.852260 3579 tablet_copy_service.cc:161] P af80abcc7a194cc582c7c45a62e49086: Beginning new tablet copy session on tablet 4cb6b3485dc345128d46837f72faa202 from peer f0f5056255ed4763b222e74191d4d582 at {username='slave'} at 127.20.210.3:59007: session id = f0f5056255ed4763b222e74191d4d582-4cb6b3485dc345128d46837f72faa202
I20240501 19:57:25.853135 3579 tablet_copy_source_session.cc:205] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:25.855409 4713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cb6b3485dc345128d46837f72faa202. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:25.855592 4707 tablet_copy_client.cc:783] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:25.855782 4707 tablet_copy_client.cc:647] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:25.857182 4707 tablet_copy_client.cc:523] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:25.858415 4707 tablet_bootstrap.cc:492] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469: Bootstrap starting.
I20240501 19:57:25.859807 4713 tablet_copy_client.cc:783] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:25.860044 4713 tablet_copy_client.cc:647] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:25.861560 4713 tablet_copy_client.cc:523] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:25.862772 4713 tablet_bootstrap.cc:492] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582: Bootstrap starting.
I20240501 19:57:25.863693 4707 log.cc:824] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:25.867219 4713 log.cc:824] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:25.880641 4707 tablet_bootstrap.cc:492] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:25.880887 4707 tablet_bootstrap.cc:492] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469: Bootstrap complete.
I20240501 19:57:25.881114 4707 ts_tablet_manager.cc:1367] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469: Time spent bootstrapping tablet: real 0.023s user 0.019s sys 0.000s
I20240501 19:57:25.883212 4707 raft_consensus.cc:348] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } }
I20240501 19:57:25.883412 4707 raft_consensus.cc:729] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c1a344780dbb4de9bcb624ac5d91d469, State: Initialized, Role: LEARNER
I20240501 19:57:25.883775 4707 consensus_queue.cc:260] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } }
I20240501 19:57:25.884531 4707 ts_tablet_manager.cc:1398] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240501 19:57:25.884688 4713 tablet_bootstrap.cc:492] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:25.885018 4713 tablet_bootstrap.cc:492] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582: Bootstrap complete.
I20240501 19:57:25.885139 3978 tablet_copy_service.cc:342] P 1d5d277450784129ab6e7b31597a48d5: Request end of tablet copy session c1a344780dbb4de9bcb624ac5d91d469-addbafe6232c4790be3235342b47bf7d received from {username='slave'} at 127.20.210.2:44929
I20240501 19:57:25.885214 3978 tablet_copy_service.cc:433] P 1d5d277450784129ab6e7b31597a48d5: ending tablet copy session c1a344780dbb4de9bcb624ac5d91d469-addbafe6232c4790be3235342b47bf7d on tablet addbafe6232c4790be3235342b47bf7d with peer c1a344780dbb4de9bcb624ac5d91d469
I20240501 19:57:25.885282 4713 ts_tablet_manager.cc:1367] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582: Time spent bootstrapping tablet: real 0.023s user 0.022s sys 0.000s
I20240501 19:57:25.887362 4713 raft_consensus.cc:348] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } }
I20240501 19:57:25.887581 4713 raft_consensus.cc:729] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f0f5056255ed4763b222e74191d4d582, State: Initialized, Role: LEARNER
I20240501 19:57:25.887766 4727 ts_tablet_manager.cc:902] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582: Initiating tablet copy from peer af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143)
I20240501 19:57:25.887873 4713 consensus_queue.cc:260] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } }
I20240501 19:57:25.888201 4727 tablet_copy_client.cc:308] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:36143
I20240501 19:57:25.888376 4462 heartbeater.cc:503] Master 127.20.210.62:40877 was elected leader, sending a full tablet report...
I20240501 19:57:25.888577 3579 tablet_copy_service.cc:140] P af80abcc7a194cc582c7c45a62e49086: Received BeginTabletCopySession request for tablet 78f3dca63e9b419ca0f55261bdd7875d from peer f0f5056255ed4763b222e74191d4d582 ({username='slave'} at 127.20.210.3:59007)
I20240501 19:57:25.888648 3579 tablet_copy_service.cc:161] P af80abcc7a194cc582c7c45a62e49086: Beginning new tablet copy session on tablet 78f3dca63e9b419ca0f55261bdd7875d from peer f0f5056255ed4763b222e74191d4d582 at {username='slave'} at 127.20.210.3:59007: session id = f0f5056255ed4763b222e74191d4d582-78f3dca63e9b419ca0f55261bdd7875d
I20240501 19:57:25.888681 4713 ts_tablet_manager.cc:1398] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20240501 19:57:25.889472 3579 tablet_copy_source_session.cc:205] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:25.890292 3579 tablet_copy_service.cc:342] P af80abcc7a194cc582c7c45a62e49086: Request end of tablet copy session f0f5056255ed4763b222e74191d4d582-4cb6b3485dc345128d46837f72faa202 received from {username='slave'} at 127.20.210.3:59007
I20240501 19:57:25.890373 3579 tablet_copy_service.cc:433] P af80abcc7a194cc582c7c45a62e49086: ending tablet copy session f0f5056255ed4763b222e74191d4d582-4cb6b3485dc345128d46837f72faa202 on tablet 4cb6b3485dc345128d46837f72faa202 with peer f0f5056255ed4763b222e74191d4d582
I20240501 19:57:25.890486 4727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78f3dca63e9b419ca0f55261bdd7875d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:25.892434 4727 tablet_copy_client.cc:783] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:25.892560 4727 tablet_copy_client.cc:647] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:25.893503 4727 tablet_copy_client.cc:523] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:25.894398 4727 tablet_bootstrap.cc:492] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582: Bootstrap starting.
I20240501 19:57:25.915455 4727 tablet_bootstrap.cc:492] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:25.915704 4727 tablet_bootstrap.cc:492] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582: Bootstrap complete.
I20240501 19:57:25.915858 4727 ts_tablet_manager.cc:1367] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582: Time spent bootstrapping tablet: real 0.021s user 0.020s sys 0.000s
I20240501 19:57:25.916224 4727 raft_consensus.cc:348] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } }
I20240501 19:57:25.916306 4727 raft_consensus.cc:729] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f0f5056255ed4763b222e74191d4d582, State: Initialized, Role: LEARNER
I20240501 19:57:25.916375 4727 consensus_queue.cc:260] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } }
I20240501 19:57:25.916597 4727 ts_tablet_manager.cc:1398] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:25.917033 3579 tablet_copy_service.cc:342] P af80abcc7a194cc582c7c45a62e49086: Request end of tablet copy session f0f5056255ed4763b222e74191d4d582-78f3dca63e9b419ca0f55261bdd7875d received from {username='slave'} at 127.20.210.3:59007
I20240501 19:57:25.917105 3579 tablet_copy_service.cc:433] P af80abcc7a194cc582c7c45a62e49086: ending tablet copy session f0f5056255ed4763b222e74191d4d582-78f3dca63e9b419ca0f55261bdd7875d on tablet 78f3dca63e9b419ca0f55261bdd7875d with peer f0f5056255ed4763b222e74191d4d582
I20240501 19:57:25.944062 4707 ts_tablet_manager.cc:902] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469: Initiating tablet copy from peer 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263)
I20240501 19:57:25.944485 4707 tablet_copy_client.cc:308] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:42263
I20240501 19:57:25.944952 3978 tablet_copy_service.cc:140] P 1d5d277450784129ab6e7b31597a48d5: Received BeginTabletCopySession request for tablet 4357ef00fc0841b2a12b214feb24f1a3 from peer c1a344780dbb4de9bcb624ac5d91d469 ({username='slave'} at 127.20.210.2:44929)
I20240501 19:57:25.945057 3978 tablet_copy_service.cc:161] P 1d5d277450784129ab6e7b31597a48d5: Beginning new tablet copy session on tablet 4357ef00fc0841b2a12b214feb24f1a3 from peer c1a344780dbb4de9bcb624ac5d91d469 at {username='slave'} at 127.20.210.2:44929: session id = c1a344780dbb4de9bcb624ac5d91d469-4357ef00fc0841b2a12b214feb24f1a3
I20240501 19:57:25.945693 3978 tablet_copy_source_session.cc:205] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:25.946434 4707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4357ef00fc0841b2a12b214feb24f1a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:25.947851 4707 tablet_copy_client.cc:783] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:25.947970 4707 tablet_copy_client.cc:647] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:25.948872 4707 tablet_copy_client.cc:523] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:25.949752 4707 tablet_bootstrap.cc:492] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469: Bootstrap starting.
I20240501 19:57:25.964985 4707 tablet_bootstrap.cc:492] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:25.965196 4707 tablet_bootstrap.cc:492] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469: Bootstrap complete.
I20240501 19:57:25.965335 4707 ts_tablet_manager.cc:1367] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.003s
I20240501 19:57:25.965725 4707 raft_consensus.cc:348] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } }
I20240501 19:57:25.965812 4707 raft_consensus.cc:729] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c1a344780dbb4de9bcb624ac5d91d469, State: Initialized, Role: LEARNER
I20240501 19:57:25.965899 4707 consensus_queue.cc:260] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } }
I20240501 19:57:25.966104 4707 ts_tablet_manager.cc:1398] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:25.966487 3978 tablet_copy_service.cc:342] P 1d5d277450784129ab6e7b31597a48d5: Request end of tablet copy session c1a344780dbb4de9bcb624ac5d91d469-4357ef00fc0841b2a12b214feb24f1a3 received from {username='slave'} at 127.20.210.2:44929
I20240501 19:57:25.966552 3978 tablet_copy_service.cc:433] P 1d5d277450784129ab6e7b31597a48d5: ending tablet copy session c1a344780dbb4de9bcb624ac5d91d469-4357ef00fc0841b2a12b214feb24f1a3 on tablet 4357ef00fc0841b2a12b214feb24f1a3 with peer c1a344780dbb4de9bcb624ac5d91d469
I20240501 19:57:25.988688 4707 ts_tablet_manager.cc:902] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469: Initiating tablet copy from peer af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143)
I20240501 19:57:25.989079 4707 tablet_copy_client.cc:308] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:36143
I20240501 19:57:25.991994 3579 tablet_copy_service.cc:140] P af80abcc7a194cc582c7c45a62e49086: Received BeginTabletCopySession request for tablet cb4bd5cf721f43838ca02bfdb3797310 from peer c1a344780dbb4de9bcb624ac5d91d469 ({username='slave'} at 127.20.210.2:32881)
I20240501 19:57:25.992086 3579 tablet_copy_service.cc:161] P af80abcc7a194cc582c7c45a62e49086: Beginning new tablet copy session on tablet cb4bd5cf721f43838ca02bfdb3797310 from peer c1a344780dbb4de9bcb624ac5d91d469 at {username='slave'} at 127.20.210.2:32881: session id = c1a344780dbb4de9bcb624ac5d91d469-cb4bd5cf721f43838ca02bfdb3797310
I20240501 19:57:25.992914 3579 tablet_copy_source_session.cc:205] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:25.993556 4707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4bd5cf721f43838ca02bfdb3797310. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:25.994884 4707 tablet_copy_client.cc:783] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:25.995007 4707 tablet_copy_client.cc:647] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:25.995900 4707 tablet_copy_client.cc:523] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:25.996717 4707 tablet_bootstrap.cc:492] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469: Bootstrap starting.
I20240501 19:57:26.001494 4727 ts_tablet_manager.cc:902] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582: Initiating tablet copy from peer af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143)
I20240501 19:57:26.001873 4727 tablet_copy_client.cc:308] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:36143
I20240501 19:57:26.002275 3579 tablet_copy_service.cc:140] P af80abcc7a194cc582c7c45a62e49086: Received BeginTabletCopySession request for tablet e47028a7fe35419b85df1e7e5791a9ff from peer f0f5056255ed4763b222e74191d4d582 ({username='slave'} at 127.20.210.3:59007)
I20240501 19:57:26.002352 3579 tablet_copy_service.cc:161] P af80abcc7a194cc582c7c45a62e49086: Beginning new tablet copy session on tablet e47028a7fe35419b85df1e7e5791a9ff from peer f0f5056255ed4763b222e74191d4d582 at {username='slave'} at 127.20.210.3:59007: session id = f0f5056255ed4763b222e74191d4d582-e47028a7fe35419b85df1e7e5791a9ff
I20240501 19:57:26.002841 3579 tablet_copy_source_session.cc:205] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:26.003531 4727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e47028a7fe35419b85df1e7e5791a9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:26.005363 4727 tablet_copy_client.cc:783] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:26.005488 4727 tablet_copy_client.cc:647] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:26.006464 4727 tablet_copy_client.cc:523] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:26.007293 4727 tablet_bootstrap.cc:492] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582: Bootstrap starting.
I20240501 19:57:26.013198 4707 tablet_bootstrap.cc:492] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:26.013406 4707 tablet_bootstrap.cc:492] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469: Bootstrap complete.
I20240501 19:57:26.013541 4707 ts_tablet_manager.cc:1367] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.003s
I20240501 19:57:26.013855 4707 raft_consensus.cc:348] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } }
I20240501 19:57:26.013939 4707 raft_consensus.cc:729] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c1a344780dbb4de9bcb624ac5d91d469, State: Initialized, Role: LEARNER
I20240501 19:57:26.014024 4707 consensus_queue.cc:260] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: true } }
I20240501 19:57:26.014290 4707 ts_tablet_manager.cc:1398] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 19:57:26.014710 3579 tablet_copy_service.cc:342] P af80abcc7a194cc582c7c45a62e49086: Request end of tablet copy session c1a344780dbb4de9bcb624ac5d91d469-cb4bd5cf721f43838ca02bfdb3797310 received from {username='slave'} at 127.20.210.2:32881
I20240501 19:57:26.014781 3579 tablet_copy_service.cc:433] P af80abcc7a194cc582c7c45a62e49086: ending tablet copy session c1a344780dbb4de9bcb624ac5d91d469-cb4bd5cf721f43838ca02bfdb3797310 on tablet cb4bd5cf721f43838ca02bfdb3797310 with peer c1a344780dbb4de9bcb624ac5d91d469
I20240501 19:57:26.023402 4727 tablet_bootstrap.cc:492] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:26.023576 4727 tablet_bootstrap.cc:492] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582: Bootstrap complete.
I20240501 19:57:26.023682 4727 ts_tablet_manager.cc:1367] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582: Time spent bootstrapping tablet: real 0.016s user 0.015s sys 0.000s
I20240501 19:57:26.024144 4727 raft_consensus.cc:348] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } }
I20240501 19:57:26.024235 4727 raft_consensus.cc:729] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f0f5056255ed4763b222e74191d4d582, State: Initialized, Role: LEARNER
I20240501 19:57:26.024338 4727 consensus_queue.cc:260] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: NON_VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: true } }
I20240501 19:57:26.024546 4727 ts_tablet_manager.cc:1398] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:26.024994 3579 tablet_copy_service.cc:342] P af80abcc7a194cc582c7c45a62e49086: Request end of tablet copy session f0f5056255ed4763b222e74191d4d582-e47028a7fe35419b85df1e7e5791a9ff received from {username='slave'} at 127.20.210.3:59007
I20240501 19:57:26.025067 3579 tablet_copy_service.cc:433] P af80abcc7a194cc582c7c45a62e49086: ending tablet copy session f0f5056255ed4763b222e74191d4d582-e47028a7fe35419b85df1e7e5791a9ff on tablet e47028a7fe35419b85df1e7e5791a9ff with peer f0f5056255ed4763b222e74191d4d582
I20240501 19:57:26.328011 4283 raft_consensus.cc:1203] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:57:26.401150 4415 raft_consensus.cc:1203] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:57:26.421027 4415 raft_consensus.cc:1203] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:57:26.453426 4283 raft_consensus.cc:1203] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:57:26.461046 4283 raft_consensus.cc:1203] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:57:26.575835 4415 raft_consensus.cc:1203] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240501 19:57:26.858734 4134 raft_consensus.cc:1050] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5: attempting to promote NON_VOTER c1a344780dbb4de9bcb624ac5d91d469 to VOTER
I20240501 19:57:26.859125 4134 consensus_queue.cc:237] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } }
I20240501 19:57:26.860329 4283 raft_consensus.cc:1261] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469 [term 1 LEARNER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:26.860335 3558 raft_consensus.cc:1261] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:26.860663 4134 consensus_queue.cc:1035] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:26.860863 4805 consensus_queue.cc:1035] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:26.862515 4134 raft_consensus.cc:2932] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.862998 3558 raft_consensus.cc:2932] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.863955 4283 raft_consensus.cc:2932] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.866281 3408 catalog_manager.cc:5484] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 reported cstate change: config changed from index 4 to 5, c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1d5d277450784129ab6e7b31597a48d5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:26.869387 3956 consensus_queue.cc:237] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } }
I20240501 19:57:26.870204 4283 raft_consensus.cc:1261] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469 [term 1 FOLLOWER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:26.870606 4805 consensus_queue.cc:1035] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:26.871783 4804 raft_consensus.cc:2932] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER af80abcc7a194cc582c7c45a62e49086 (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.872186 4283 raft_consensus.cc:2932] T addbafe6232c4790be3235342b47bf7d P c1a344780dbb4de9bcb624ac5d91d469 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER af80abcc7a194cc582c7c45a62e49086 (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.873059 3402 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet addbafe6232c4790be3235342b47bf7d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:26.873878 3410 catalog_manager.cc:5484] T addbafe6232c4790be3235342b47bf7d P 1d5d277450784129ab6e7b31597a48d5 reported cstate change: config changed from index 5 to 6, VOTER af80abcc7a194cc582c7c45a62e49086 (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "1d5d277450784129ab6e7b31597a48d5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:26.875758 3537 tablet_service.cc:1514] Processing DeleteTablet for tablet addbafe6232c4790be3235342b47bf7d with delete_type TABLET_DATA_TOMBSTONED (TS af80abcc7a194cc582c7c45a62e49086 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42722
I20240501 19:57:26.876113 4906 tablet_replica.cc:330] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086: stopping tablet replica
I20240501 19:57:26.876381 4906 raft_consensus.cc:2229] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:26.876554 4906 raft_consensus.cc:2258] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:26.877492 4906 ts_tablet_manager.cc:1875] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:26.879191 4906 ts_tablet_manager.cc:1888] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:57:26.879264 4906 log.cc:1196] T addbafe6232c4790be3235342b47bf7d P af80abcc7a194cc582c7c45a62e49086: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/addbafe6232c4790be3235342b47bf7d
I20240501 19:57:26.879762 3401 catalog_manager.cc:4830] TS af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143): tablet addbafe6232c4790be3235342b47bf7d (table rebalance_test_table_1 [id=51c847e5c2bd4a7c8e47906ec8359881]) successfully deleted
I20240501 19:57:26.923679 4616 raft_consensus.cc:1050] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086: attempting to promote NON_VOTER f0f5056255ed4763b222e74191d4d582 to VOTER
I20240501 19:57:26.924085 4616 consensus_queue.cc:237] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } }
I20240501 19:57:26.925176 3956 raft_consensus.cc:1261] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:26.925249 4415 raft_consensus.cc:1261] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582 [term 1 LEARNER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:26.925552 4838 consensus_queue.cc:1035] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:26.925709 4837 consensus_queue.cc:1035] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:26.927275 4837 raft_consensus.cc:2932] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f0f5056255ed4763b222e74191d4d582 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:26.927681 3956 raft_consensus.cc:2932] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f0f5056255ed4763b222e74191d4d582 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:26.929422 3410 catalog_manager.cc:5484] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 reported cstate change: config changed from index 4 to 5, f0f5056255ed4763b222e74191d4d582 (127.20.210.3) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:26.929461 4415 raft_consensus.cc:2932] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f0f5056255ed4763b222e74191d4d582 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:26.933202 3558 consensus_queue.cc:237] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } }
I20240501 19:57:26.934033 4415 raft_consensus.cc:1261] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:26.934402 4616 consensus_queue.cc:1035] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:26.935631 4838 raft_consensus.cc:2932] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:26.936026 4415 raft_consensus.cc:2932] T 4cb6b3485dc345128d46837f72faa202 P f0f5056255ed4763b222e74191d4d582 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:26.936852 3401 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4cb6b3485dc345128d46837f72faa202 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:26.937541 3414 catalog_manager.cc:5484] T 4cb6b3485dc345128d46837f72faa202 P af80abcc7a194cc582c7c45a62e49086 reported cstate change: config changed from index 5 to 6, VOTER 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:26.939405 3936 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cb6b3485dc345128d46837f72faa202 with delete_type TABLET_DATA_TOMBSTONED (TS 1d5d277450784129ab6e7b31597a48d5 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:33184
I20240501 19:57:26.939837 4910 tablet_replica.cc:330] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5: stopping tablet replica
I20240501 19:57:26.940086 4910 raft_consensus.cc:2229] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:26.940253 4910 raft_consensus.cc:2258] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:26.941259 4910 ts_tablet_manager.cc:1875] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:26.942979 4910 ts_tablet_manager.cc:1888] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:57:26.943069 4910 log.cc:1196] T 4cb6b3485dc345128d46837f72faa202 P 1d5d277450784129ab6e7b31597a48d5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/4cb6b3485dc345128d46837f72faa202
I20240501 19:57:26.943631 4616 raft_consensus.cc:1050] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086: attempting to promote NON_VOTER c1a344780dbb4de9bcb624ac5d91d469 to VOTER
I20240501 19:57:26.943686 3402 catalog_manager.cc:4830] TS 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263): tablet 4cb6b3485dc345128d46837f72faa202 (table rebalance_test_table_0 [id=82ab925b1e4341748da3706c139a7474]) successfully deleted
I20240501 19:57:26.943987 4616 consensus_queue.cc:237] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } }
I20240501 19:57:26.944972 4283 raft_consensus.cc:1261] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469 [term 1 LEARNER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:26.945034 3956 raft_consensus.cc:1261] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:26.945343 4838 consensus_queue.cc:1035] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:26.945502 4616 consensus_queue.cc:1035] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:26.946961 4837 raft_consensus.cc:2932] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.947372 3956 raft_consensus.cc:2932] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.947367 4283 raft_consensus.cc:2932] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.948969 3408 catalog_manager.cc:5484] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 reported cstate change: config changed from index 4 to 5, c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.966034 4837 raft_consensus.cc:1050] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086: attempting to promote NON_VOTER f0f5056255ed4763b222e74191d4d582 to VOTER
I20240501 19:57:26.966342 4837 consensus_queue.cc:237] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } }
I20240501 19:57:26.967294 4415 raft_consensus.cc:1261] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582 [term 1 LEARNER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:26.967332 3956 raft_consensus.cc:1261] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:26.967624 4838 consensus_queue.cc:1035] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:26.967794 4616 consensus_queue.cc:1035] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:26.969214 4616 raft_consensus.cc:2932] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f0f5056255ed4763b222e74191d4d582 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:26.969619 3956 raft_consensus.cc:2932] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f0f5056255ed4763b222e74191d4d582 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:26.969707 4415 raft_consensus.cc:2932] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f0f5056255ed4763b222e74191d4d582 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:26.971431 3410 catalog_manager.cc:5484] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 reported cstate change: config changed from index 4 to 5, f0f5056255ed4763b222e74191d4d582 (127.20.210.3) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:26.973824 3558 consensus_queue.cc:237] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } }
I20240501 19:57:26.974706 4415 raft_consensus.cc:1261] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:26.975000 4837 consensus_queue.cc:1035] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:26.976102 4838 raft_consensus.cc:2932] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:26.976477 4415 raft_consensus.cc:2932] T e47028a7fe35419b85df1e7e5791a9ff P f0f5056255ed4763b222e74191d4d582 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:26.977300 3401 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e47028a7fe35419b85df1e7e5791a9ff with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:26.978166 3408 catalog_manager.cc:5484] T e47028a7fe35419b85df1e7e5791a9ff P af80abcc7a194cc582c7c45a62e49086 reported cstate change: config changed from index 5 to 6, VOTER 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:26.980077 3936 tablet_service.cc:1514] Processing DeleteTablet for tablet e47028a7fe35419b85df1e7e5791a9ff with delete_type TABLET_DATA_TOMBSTONED (TS 1d5d277450784129ab6e7b31597a48d5 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:33184
I20240501 19:57:26.980276 4910 tablet_replica.cc:330] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5: stopping tablet replica
I20240501 19:57:26.980410 4910 raft_consensus.cc:2229] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:26.980544 4910 raft_consensus.cc:2258] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:26.981098 4910 ts_tablet_manager.cc:1875] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:26.981923 4805 raft_consensus.cc:1050] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5: attempting to promote NON_VOTER c1a344780dbb4de9bcb624ac5d91d469 to VOTER
I20240501 19:57:26.982151 4805 consensus_queue.cc:237] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } }
I20240501 19:57:26.982782 4910 ts_tablet_manager.cc:1888] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:57:26.982877 4910 log.cc:1196] T e47028a7fe35419b85df1e7e5791a9ff P 1d5d277450784129ab6e7b31597a48d5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/e47028a7fe35419b85df1e7e5791a9ff
I20240501 19:57:26.983115 3558 raft_consensus.cc:1261] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:26.983245 4283 raft_consensus.cc:1261] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469 [term 1 LEARNER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:26.983424 3402 catalog_manager.cc:4830] TS 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263): tablet e47028a7fe35419b85df1e7e5791a9ff (table rebalance_test_table_1 [id=51c847e5c2bd4a7c8e47906ec8359881]) successfully deleted
I20240501 19:57:26.983461 4805 consensus_queue.cc:1035] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:26.983882 4805 consensus_queue.cc:1035] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:26.985041 4804 raft_consensus.cc:2932] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.985430 3558 raft_consensus.cc:2932] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.986930 3414 catalog_manager.cc:5484] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 reported cstate change: config changed from index 4 to 5, c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1d5d277450784129ab6e7b31597a48d5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:26.986966 4282 raft_consensus.cc:2932] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.990496 3956 consensus_queue.cc:237] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } }
I20240501 19:57:26.991307 4282 raft_consensus.cc:1261] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469 [term 1 FOLLOWER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:26.991622 4804 consensus_queue.cc:1035] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:26.992839 4134 raft_consensus.cc:2932] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER af80abcc7a194cc582c7c45a62e49086 (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.993165 4282 raft_consensus.cc:2932] T 4357ef00fc0841b2a12b214feb24f1a3 P c1a344780dbb4de9bcb624ac5d91d469 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER af80abcc7a194cc582c7c45a62e49086 (127.20.210.1) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:26.993810 3402 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4357ef00fc0841b2a12b214feb24f1a3 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:26.994907 3408 catalog_manager.cc:5484] T 4357ef00fc0841b2a12b214feb24f1a3 P 1d5d277450784129ab6e7b31597a48d5 reported cstate change: config changed from index 5 to 6, VOTER af80abcc7a194cc582c7c45a62e49086 (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "1d5d277450784129ab6e7b31597a48d5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:26.996595 3537 tablet_service.cc:1514] Processing DeleteTablet for tablet 4357ef00fc0841b2a12b214feb24f1a3 with delete_type TABLET_DATA_TOMBSTONED (TS af80abcc7a194cc582c7c45a62e49086 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42722
I20240501 19:57:26.996757 4906 tablet_replica.cc:330] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086: stopping tablet replica
I20240501 19:57:26.996888 4906 raft_consensus.cc:2229] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:26.997020 4906 raft_consensus.cc:2258] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:26.997560 4906 ts_tablet_manager.cc:1875] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:26.999047 4906 ts_tablet_manager.cc:1888] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:57:26.999135 4906 log.cc:1196] T 4357ef00fc0841b2a12b214feb24f1a3 P af80abcc7a194cc582c7c45a62e49086: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/4357ef00fc0841b2a12b214feb24f1a3
I20240501 19:57:26.999557 3401 catalog_manager.cc:4830] TS af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143): tablet 4357ef00fc0841b2a12b214feb24f1a3 (table rebalance_test_table_0 [id=82ab925b1e4341748da3706c139a7474]) successfully deleted
I20240501 19:57:27.032011 4838 raft_consensus.cc:1050] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086: attempting to promote NON_VOTER f0f5056255ed4763b222e74191d4d582 to VOTER
I20240501 19:57:27.032328 4838 consensus_queue.cc:237] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } }
I20240501 19:57:27.033414 4415 raft_consensus.cc:1261] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582 [term 1 LEARNER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:27.033493 3956 raft_consensus.cc:1261] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:27.033790 4837 consensus_queue.cc:1035] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:27.033954 4838 consensus_queue.cc:1035] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:27.035719 4616 raft_consensus.cc:2932] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f0f5056255ed4763b222e74191d4d582 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:27.036139 4415 raft_consensus.cc:2932] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f0f5056255ed4763b222e74191d4d582 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:27.037395 3956 raft_consensus.cc:2932] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f0f5056255ed4763b222e74191d4d582 (127.20.210.3) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:27.038456 3410 catalog_manager.cc:5484] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582 reported cstate change: config changed from index 4 to 5, f0f5056255ed4763b222e74191d4d582 (127.20.210.3) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } attrs { replace: true } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:27.041249 3558 consensus_queue.cc:237] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } }
I20240501 19:57:27.042512 4415 raft_consensus.cc:1261] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582 [term 1 FOLLOWER]: Refusing update from remote peer af80abcc7a194cc582c7c45a62e49086: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:27.042860 4926 consensus_queue.cc:1035] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:27.044370 4838 raft_consensus.cc:2932] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:27.044872 4415 raft_consensus.cc:2932] T 78f3dca63e9b419ca0f55261bdd7875d P f0f5056255ed4763b222e74191d4d582 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } } }
I20240501 19:57:27.045552 3401 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 78f3dca63e9b419ca0f55261bdd7875d with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:27.046905 3408 catalog_manager.cc:5484] T 78f3dca63e9b419ca0f55261bdd7875d P af80abcc7a194cc582c7c45a62e49086 reported cstate change: config changed from index 5 to 6, VOTER 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "af80abcc7a194cc582c7c45a62e49086" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0f5056255ed4763b222e74191d4d582" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 38243 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:27.048821 3936 tablet_service.cc:1514] Processing DeleteTablet for tablet 78f3dca63e9b419ca0f55261bdd7875d with delete_type TABLET_DATA_TOMBSTONED (TS 1d5d277450784129ab6e7b31597a48d5 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:33184
I20240501 19:57:27.048993 4910 tablet_replica.cc:330] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5: stopping tablet replica
I20240501 19:57:27.049130 4910 raft_consensus.cc:2229] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:27.049268 4910 raft_consensus.cc:2258] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:27.049834 4910 ts_tablet_manager.cc:1875] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:27.051548 4910 ts_tablet_manager.cc:1888] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240501 19:57:27.051631 4910 log.cc:1196] T 78f3dca63e9b419ca0f55261bdd7875d P 1d5d277450784129ab6e7b31597a48d5: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/78f3dca63e9b419ca0f55261bdd7875d
I20240501 19:57:27.052107 3402 catalog_manager.cc:4830] TS 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4:42263): tablet 78f3dca63e9b419ca0f55261bdd7875d (table rebalance_test_table_2 [id=adfec7d1f5ff4471ad6da4c37fffa367]) successfully deleted
I20240501 19:57:27.062614 3558 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cb4bd5cf721f43838ca02bfdb3797310"
dest_uuid: "af80abcc7a194cc582c7c45a62e49086"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43098
I20240501 19:57:27.062729 3558 raft_consensus.cc:595] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:57:27.080672 3558 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cb4bd5cf721f43838ca02bfdb3797310"
dest_uuid: "af80abcc7a194cc582c7c45a62e49086"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43128
I20240501 19:57:27.080762 3558 raft_consensus.cc:595] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:57:27.290812 3558 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cb4bd5cf721f43838ca02bfdb3797310"
dest_uuid: "af80abcc7a194cc582c7c45a62e49086"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43144
I20240501 19:57:27.290933 3558 raft_consensus.cc:595] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:57:27.372455 4838 raft_consensus.cc:979] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086: : Instructing follower 1d5d277450784129ab6e7b31597a48d5 to start an election
I20240501 19:57:27.372575 4838 raft_consensus.cc:1067] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Signalling peer 1d5d277450784129ab6e7b31597a48d5 to start an election
I20240501 19:57:27.373096 3956 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cb4bd5cf721f43838ca02bfdb3797310"
dest_uuid: "1d5d277450784129ab6e7b31597a48d5"
from {username='slave'} at 127.20.210.1:38751
I20240501 19:57:27.373281 3956 raft_consensus.cc:482] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 19:57:27.373353 3956 raft_consensus.cc:3037] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:27.374156 3956 raft_consensus.cc:504] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } }
I20240501 19:57:27.374514 3956 leader_election.cc:288] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 2 election: Requested vote from peers af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143), c1a344780dbb4de9bcb624ac5d91d469 (127.20.210.2:34623)
I20240501 19:57:27.374845 3558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4bd5cf721f43838ca02bfdb3797310" candidate_uuid: "1d5d277450784129ab6e7b31597a48d5" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "af80abcc7a194cc582c7c45a62e49086"
I20240501 19:57:27.374965 3558 raft_consensus.cc:3032] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 19:57:27.375010 3558 raft_consensus.cc:729] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: af80abcc7a194cc582c7c45a62e49086, State: Running, Role: LEADER
I20240501 19:57:27.375001 4282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4bd5cf721f43838ca02bfdb3797310" candidate_uuid: "1d5d277450784129ab6e7b31597a48d5" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "c1a344780dbb4de9bcb624ac5d91d469"
I20240501 19:57:27.375190 4282 raft_consensus.cc:3037] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:27.375185 3558 consensus_queue.cc:260] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } }
I20240501 19:57:27.375413 3558 raft_consensus.cc:3037] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:27.376056 4282 raft_consensus.cc:2454] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d5d277450784129ab6e7b31597a48d5 in term 2.
I20240501 19:57:27.376171 3558 raft_consensus.cc:2454] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d5d277450784129ab6e7b31597a48d5 in term 2.
I20240501 19:57:27.376361 3893 leader_election.cc:302] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d5d277450784129ab6e7b31597a48d5, c1a344780dbb4de9bcb624ac5d91d469; no voters:
I20240501 19:57:27.376528 4134 raft_consensus.cc:2781] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 2 FOLLOWER]: Leader election won for term 2
I20240501 19:57:27.376623 4134 raft_consensus.cc:686] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 2 LEADER]: Becoming Leader. State: Replica: 1d5d277450784129ab6e7b31597a48d5, State: Running, Role: LEADER
I20240501 19:57:27.376785 4134 consensus_queue.cc:237] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } }
I20240501 19:57:27.378159 3414 catalog_manager.cc:5484] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 reported cstate change: term changed from 1 to 2, leader changed from af80abcc7a194cc582c7c45a62e49086 (127.20.210.1) to 1d5d277450784129ab6e7b31597a48d5 (127.20.210.4). New cstate: current_term: 2 leader_uuid: "1d5d277450784129ab6e7b31597a48d5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:27.907753 4282 raft_consensus.cc:1261] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469 [term 2 FOLLOWER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 19:57:27.908299 4134 consensus_queue.cc:1035] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:27.911504 3558 raft_consensus.cc:1261] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 2 FOLLOWER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240501 19:57:27.912055 4996 consensus_queue.cc:1035] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af80abcc7a194cc582c7c45a62e49086" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 36143 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:27.913547 3956 consensus_queue.cc:237] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } }
I20240501 19:57:27.914408 4282 raft_consensus.cc:1261] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469 [term 2 FOLLOWER]: Refusing update from remote peer 1d5d277450784129ab6e7b31597a48d5: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240501 19:57:27.914709 4134 consensus_queue.cc:1035] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:27.916050 4998 raft_consensus.cc:2932] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER af80abcc7a194cc582c7c45a62e49086 (127.20.210.1) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:27.916432 4282 raft_consensus.cc:2932] T cb4bd5cf721f43838ca02bfdb3797310 P c1a344780dbb4de9bcb624ac5d91d469 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER af80abcc7a194cc582c7c45a62e49086 (127.20.210.1) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } } }
I20240501 19:57:27.917622 3402 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet cb4bd5cf721f43838ca02bfdb3797310 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:27.918157 3414 catalog_manager.cc:5484] T cb4bd5cf721f43838ca02bfdb3797310 P 1d5d277450784129ab6e7b31597a48d5 reported cstate change: config changed from index 5 to 7, VOTER af80abcc7a194cc582c7c45a62e49086 (127.20.210.1) evicted. New cstate: current_term: 2 leader_uuid: "1d5d277450784129ab6e7b31597a48d5" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1d5d277450784129ab6e7b31597a48d5" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1a344780dbb4de9bcb624ac5d91d469" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 34623 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:27.920776 3537 tablet_service.cc:1514] Processing DeleteTablet for tablet cb4bd5cf721f43838ca02bfdb3797310 with delete_type TABLET_DATA_TOMBSTONED (TS af80abcc7a194cc582c7c45a62e49086 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:42722
I20240501 19:57:27.921063 5005 tablet_replica.cc:330] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086: stopping tablet replica
I20240501 19:57:27.921195 5005 raft_consensus.cc:2229] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:27.921355 5005 raft_consensus.cc:2258] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:27.921885 5005 ts_tablet_manager.cc:1875] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:27.923753 5005 ts_tablet_manager.cc:1888] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240501 19:57:27.923857 5005 log.cc:1196] T cb4bd5cf721f43838ca02bfdb3797310 P af80abcc7a194cc582c7c45a62e49086: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/cb4bd5cf721f43838ca02bfdb3797310
I20240501 19:57:27.924281 3401 catalog_manager.cc:4830] TS af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143): tablet cb4bd5cf721f43838ca02bfdb3797310 (table rebalance_test_table_2 [id=adfec7d1f5ff4471ad6da4c37fffa367]) successfully deleted
I20240501 19:57:28.170356 4394 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:28.172955 3936 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:57:28.173909 4263 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:28.175065 3537 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:28.255199 3537 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:28.255695 4263 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:28.256062 4394 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:28.256860 3936 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:57:28.323699 3936 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240501 19:57:28.323952 3537 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:28.325136 4263 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:28.325157 4394 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:28.354336 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 3476
I20240501 19:57:28.360298 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 4201
I20240501 19:57:28.365319 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 4333
W20240501 19:57:28.366914 3892 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.210.1:36143: connect: Connection refused (error 111)
W20240501 19:57:28.367664 3892 consensus_peers.cc:485] T 931e609c374a46658347e2da5dcc0f9d P 1d5d277450784129ab6e7b31597a48d5 -> Peer af80abcc7a194cc582c7c45a62e49086 (127.20.210.1:36143): Couldn't send request to peer af80abcc7a194cc582c7c45a62e49086. Status: Network error: Client connection negotiation failed: client connection to 127.20.210.1:36143: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:28.370654 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 3874
I20240501 19:57:28.376060 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 3384
2024-05-01T19:57:28Z chronyd exiting
[ OK ] LocationAwareRebalancingParamTest.Rebalance/idx7_rf2_t3_l_1_2_1 (10134 ms)
[ RUN ] LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer/idx6_rf7_t7_l_3_4_3
I20240501 19:57:28.399528 21320 test_util.cc:268] Using random seed: -445836635
I20240501 19:57:28.400244 21320 ts_itest-base.cc:112] Starting cluster with:
I20240501 19:57:28.400292 21320 ts_itest-base.cc:113] --------------
I20240501 19:57:28.400313 21320 ts_itest-base.cc:114] 10 tablet servers
I20240501 19:57:28.400328 21320 ts_itest-base.cc:115] 7 replicas per TS
I20240501 19:57:28.400357 21320 ts_itest-base.cc:116] --------------
2024-05-01T19:57:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-01T19:57:28Z Disabled control of system clock
I20240501 19:57:28.407552 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:45969
--webserver_interface=127.20.210.62
--webserver_port=0
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.210.62:45969
--location_mapping_cmd=/tmp/dist-test-taskNtLy6c/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/location-assignment.state --map /A:3 --map /B:4 --map /C:3
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false
--master_client_location_assignment_enabled=false
--allow_unsafe_replication_factor with env {}
W20240501 19:57:28.501744 5122 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240501 19:57:28.501941 5122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:28.501991 5122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:28.502051 5122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:28.504738 5122 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:28.504815 5122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240501 19:57:28.504861 5122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:28.504895 5122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240501 19:57:28.504920 5122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240501 19:57:28.508136 5122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--location_mapping_cmd=/tmp/dist-test-taskNtLy6c/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/location-assignment.state --map /A:3 --map /B:4 --map /C:3
--ipki_ca_key_size=768
--master_addresses=127.20.210.62:45969
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.210.62:45969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.20.210.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.5122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:28.508853 5122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:28.509824 5122 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:28.516512 5128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:28.517242 5127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:28.517937 5130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:28.518764 5122 server_base.cc:1034] running on GCE node
I20240501 19:57:28.519147 5122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:28.519951 5122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:28.521090 5122 hybrid_clock.cc:648] HybridClock initialized: now 1714593448521075 us; error 26 us; skew 500 ppm
I20240501 19:57:28.522698 5122 webserver.cc:460] Webserver started at http://127.20.210.62:37771/ using document root <none> and password file <none>
I20240501 19:57:28.523181 5122 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:28.523238 5122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:28.523387 5122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:28.524897 5122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "5f16b54388c8410a8d44bbc221ccf145"
format_stamp: "Formatted at 2024-05-01 19:57:28 on dist-test-slave-vgh4"
I20240501 19:57:28.525352 5122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "5f16b54388c8410a8d44bbc221ccf145"
format_stamp: "Formatted at 2024-05-01 19:57:28 on dist-test-slave-vgh4"
I20240501 19:57:28.528405 5122 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 19:57:28.530310 5137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:28.531154 5122 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:28.531251 5122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
uuid: "5f16b54388c8410a8d44bbc221ccf145"
format_stamp: "Formatted at 2024-05-01 19:57:28 on dist-test-slave-vgh4"
I20240501 19:57:28.531322 5122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:28.543797 5122 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:28.544252 5122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:28.544389 5122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:28.549615 5122 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.62:45969
I20240501 19:57:28.549623 5189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.62:45969 every 8 connection(s)
I20240501 19:57:28.550449 5122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240501 19:57:28.551508 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 5122
I20240501 19:57:28.551630 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240501 19:57:28.552924 5190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:28.557994 5190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145: Bootstrap starting.
I20240501 19:57:28.560040 5190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:28.560678 5190 log.cc:824] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:28.561966 5190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145: No bootstrap required, opened a new log
I20240501 19:57:28.564201 5190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f16b54388c8410a8d44bbc221ccf145" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 45969 } }
I20240501 19:57:28.564374 5190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:28.564442 5190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f16b54388c8410a8d44bbc221ccf145, State: Initialized, Role: FOLLOWER
I20240501 19:57:28.564805 5190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f16b54388c8410a8d44bbc221ccf145" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 45969 } }
I20240501 19:57:28.564927 5190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240501 19:57:28.564996 5190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240501 19:57:28.565089 5190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:28.565672 5190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f16b54388c8410a8d44bbc221ccf145" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 45969 } }
I20240501 19:57:28.565963 5190 leader_election.cc:302] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f16b54388c8410a8d44bbc221ccf145; no voters:
I20240501 19:57:28.566180 5190 leader_election.cc:288] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240501 19:57:28.566444 5195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:28.566620 5195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [term 1 LEADER]: Becoming Leader. State: Replica: 5f16b54388c8410a8d44bbc221ccf145, State: Running, Role: LEADER
I20240501 19:57:28.566903 5195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f16b54388c8410a8d44bbc221ccf145" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 45969 } }
I20240501 19:57:28.567005 5190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [sys.catalog]: configured and running, proceeding with master startup.
I20240501 19:57:28.568015 5197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f16b54388c8410a8d44bbc221ccf145. Latest consensus state: current_term: 1 leader_uuid: "5f16b54388c8410a8d44bbc221ccf145" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f16b54388c8410a8d44bbc221ccf145" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 45969 } } }
I20240501 19:57:28.568120 5197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [sys.catalog]: This master's current role is: LEADER
I20240501 19:57:28.568392 5196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f16b54388c8410a8d44bbc221ccf145" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f16b54388c8410a8d44bbc221ccf145" member_type: VOTER last_known_addr { host: "127.20.210.62" port: 45969 } } }
I20240501 19:57:28.568452 5196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145 [sys.catalog]: This master's current role is: LEADER
W20240501 19:57:28.570878 5210 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240501 19:57:28.570940 5210 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240501 19:57:28.571015 5205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240501 19:57:28.571693 5205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240501 19:57:28.575349 5205 catalog_manager.cc:1347] Generated new cluster ID: cfdfa48cb06740c8a44a97d630c40cae
I20240501 19:57:28.575404 5205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240501 19:57:28.589543 5205 catalog_manager.cc:1370] Generated new certificate authority record
I20240501 19:57:28.590618 5205 catalog_manager.cc:1504] Loading token signing keys...
I20240501 19:57:28.599128 5205 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 5f16b54388c8410a8d44bbc221ccf145: Generated new TSK 0
I20240501 19:57:28.599844 5205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240501 19:57:28.605967 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.1:0
--local_ip_for_outbound_sockets=127.20.210.1
--webserver_interface=127.20.210.1
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 19:57:28.698712 5214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:28.698868 5214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:28.698904 5214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:28.701907 5214 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:28.701965 5214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:28.702046 5214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.1
I20240501 19:57:28.705435 5214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.20.210.1
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--never_fsync=true
--heap_profile_path=/tmp/kudu.5214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.1
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:28.706156 5214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:28.707075 5214 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:28.714334 5223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:28.715075 5219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:28.715761 5220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:28.716436 5214 server_base.cc:1034] running on GCE node
I20240501 19:57:28.716719 5214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:28.717118 5214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:28.718263 5214 hybrid_clock.cc:648] HybridClock initialized: now 1714593448718240 us; error 37 us; skew 500 ppm
I20240501 19:57:28.719843 5214 webserver.cc:460] Webserver started at http://127.20.210.1:42845/ using document root <none> and password file <none>
I20240501 19:57:28.720297 5214 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:28.720347 5214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:28.720492 5214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:28.721814 5214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "ad52038f1d38489ea464a8fbc72daac0"
format_stamp: "Formatted at 2024-05-01 19:57:28 on dist-test-slave-vgh4"
I20240501 19:57:28.722173 5214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "ad52038f1d38489ea464a8fbc72daac0"
format_stamp: "Formatted at 2024-05-01 19:57:28 on dist-test-slave-vgh4"
I20240501 19:57:28.724987 5214 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:28.726795 5229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:28.727617 5214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:28.727706 5214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "ad52038f1d38489ea464a8fbc72daac0"
format_stamp: "Formatted at 2024-05-01 19:57:28 on dist-test-slave-vgh4"
I20240501 19:57:28.727777 5214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:28.744071 5214 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:28.744553 5214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:28.744688 5214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:28.745141 5214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:28.745860 5214 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:28.745903 5214 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:28.745940 5214 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:28.745967 5214 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:28.753461 5214 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.1:46285
I20240501 19:57:28.753491 5342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.1:46285 every 8 connection(s)
I20240501 19:57:28.754202 5214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240501 19:57:28.760723 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 5214
I20240501 19:57:28.760843 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240501 19:57:28.761843 5343 heartbeater.cc:347] Connected to a master server at 127.20.210.62:45969
I20240501 19:57:28.762040 5343 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:28.762579 5343 heartbeater.cc:511] Master 127.20.210.62:45969 requested a full tablet report, sending...
I20240501 19:57:28.763078 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.2:0
--local_ip_for_outbound_sockets=127.20.210.2
--webserver_interface=127.20.210.2
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:28.806509 5154 ts_manager.cc:194] Registered new tserver with Master: ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285)
I20240501 19:57:28.808104 5154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.1:45115
W20240501 19:57:28.871122 5346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:28.871286 5346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:28.871346 5346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:28.873859 5346 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:28.873915 5346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:28.873998 5346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.2
I20240501 19:57:28.876947 5346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.20.210.2
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--never_fsync=true
--heap_profile_path=/tmp/kudu.5346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.2
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:28.877647 5346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:28.878583 5346 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:28.886379 5352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:28.887087 5355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:28.887624 5353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:28.888374 5346 server_base.cc:1034] running on GCE node
I20240501 19:57:28.888687 5346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:28.889159 5346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:28.890295 5346 hybrid_clock.cc:648] HybridClock initialized: now 1714593448890295 us; error 31 us; skew 500 ppm
I20240501 19:57:28.891912 5346 webserver.cc:460] Webserver started at http://127.20.210.2:39497/ using document root <none> and password file <none>
I20240501 19:57:28.892377 5346 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:28.892428 5346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:28.892596 5346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:28.893985 5346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "9782508a47ba4403a39ca49fff7677fa"
format_stamp: "Formatted at 2024-05-01 19:57:28 on dist-test-slave-vgh4"
I20240501 19:57:28.894420 5346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "9782508a47ba4403a39ca49fff7677fa"
format_stamp: "Formatted at 2024-05-01 19:57:28 on dist-test-slave-vgh4"
I20240501 19:57:28.897158 5346 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:28.899070 5362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:28.899976 5346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:28.900058 5346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "9782508a47ba4403a39ca49fff7677fa"
format_stamp: "Formatted at 2024-05-01 19:57:28 on dist-test-slave-vgh4"
I20240501 19:57:28.900141 5346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:28.926021 5346 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:28.926496 5346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:28.926609 5346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:28.926996 5346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:28.927666 5346 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:28.927706 5346 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:28.927738 5346 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:28.927765 5346 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:28.935751 5346 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.2:35469
I20240501 19:57:28.935777 5475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.2:35469 every 8 connection(s)
I20240501 19:57:28.936499 5346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240501 19:57:28.938360 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 5346
I20240501 19:57:28.938486 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240501 19:57:28.940729 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.3:0
--local_ip_for_outbound_sockets=127.20.210.3
--webserver_interface=127.20.210.3
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:28.945032 5476 heartbeater.cc:347] Connected to a master server at 127.20.210.62:45969
I20240501 19:57:28.945200 5476 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:28.945616 5476 heartbeater.cc:511] Master 127.20.210.62:45969 requested a full tablet report, sending...
I20240501 19:57:28.980279 5154 ts_manager.cc:194] Registered new tserver with Master: 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
I20240501 19:57:28.980960 5154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.2:37209
W20240501 19:57:29.037070 5479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:29.037238 5479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:29.037300 5479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:29.040032 5479 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:29.040100 5479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:29.040189 5479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.3
I20240501 19:57:29.043736 5479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.20.210.3
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--never_fsync=true
--heap_profile_path=/tmp/kudu.5479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.3
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:29.044457 5479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:29.045398 5479 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:29.052616 5488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:29.053221 5486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:29.053944 5485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:29.054657 5479 server_base.cc:1034] running on GCE node
I20240501 19:57:29.054917 5479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:29.055352 5479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:29.056490 5479 hybrid_clock.cc:648] HybridClock initialized: now 1714593449056487 us; error 25 us; skew 500 ppm
I20240501 19:57:29.058025 5479 webserver.cc:460] Webserver started at http://127.20.210.3:46805/ using document root <none> and password file <none>
I20240501 19:57:29.058552 5479 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:29.058609 5479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:29.058768 5479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:29.060144 5479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "f5287d5ae4d84bd692b0f7985cd94171"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.060595 5479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "f5287d5ae4d84bd692b0f7985cd94171"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.063578 5479 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20240501 19:57:29.065515 5495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.066399 5479 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:29.066502 5479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "f5287d5ae4d84bd692b0f7985cd94171"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.066574 5479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:29.078426 5479 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:29.078881 5479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:29.079017 5479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:29.079492 5479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:29.080258 5479 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:29.080312 5479 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.080377 5479 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:29.080407 5479 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.087813 5479 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.3:44275
I20240501 19:57:29.087857 5608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.3:44275 every 8 connection(s)
I20240501 19:57:29.088595 5479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240501 19:57:29.095820 5609 heartbeater.cc:347] Connected to a master server at 127.20.210.62:45969
I20240501 19:57:29.095863 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 5479
I20240501 19:57:29.095968 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240501 19:57:29.095988 5609 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:29.096387 5609 heartbeater.cc:511] Master 127.20.210.62:45969 requested a full tablet report, sending...
I20240501 19:57:29.097782 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.4:0
--local_ip_for_outbound_sockets=127.20.210.4
--webserver_interface=127.20.210.4
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:29.128288 5154 ts_manager.cc:194] Registered new tserver with Master: f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:29.128885 5154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.3:34899
W20240501 19:57:29.200821 5613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:29.200981 5613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:29.201020 5613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:29.203831 5613 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:29.203889 5613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:29.203989 5613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.4
I20240501 19:57:29.207253 5613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.4:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.20.210.4
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--never_fsync=true
--heap_profile_path=/tmp/kudu.5613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.4
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:29.207949 5613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:29.208834 5613 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:29.215952 5618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:29.216615 5619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:29.217378 5621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:29.218173 5613 server_base.cc:1034] running on GCE node
I20240501 19:57:29.218521 5613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:29.218950 5613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:29.220098 5613 hybrid_clock.cc:648] HybridClock initialized: now 1714593449220092 us; error 24 us; skew 500 ppm
I20240501 19:57:29.221690 5613 webserver.cc:460] Webserver started at http://127.20.210.4:42645/ using document root <none> and password file <none>
I20240501 19:57:29.222148 5613 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:29.222199 5613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:29.222364 5613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:29.223800 5613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "4d4855c096ac4e1e936abf15834332ae"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.224216 5613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "4d4855c096ac4e1e936abf15834332ae"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.227123 5613 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20240501 19:57:29.228891 5627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.229745 5613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:29.229837 5613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "4d4855c096ac4e1e936abf15834332ae"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.229926 5613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:29.260215 5613 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:29.260762 5613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:29.260915 5613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:29.261350 5613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:29.262073 5613 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:29.262115 5613 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.262166 5613 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:29.262193 5613 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.269315 5613 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.4:46519
I20240501 19:57:29.269367 5741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.4:46519 every 8 connection(s)
I20240501 19:57:29.270143 5613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240501 19:57:29.276559 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 5613
I20240501 19:57:29.276664 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240501 19:57:29.277314 5742 heartbeater.cc:347] Connected to a master server at 127.20.210.62:45969
I20240501 19:57:29.277495 5742 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:29.277948 5742 heartbeater.cc:511] Master 127.20.210.62:45969 requested a full tablet report, sending...
I20240501 19:57:29.278546 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.5:0
--local_ip_for_outbound_sockets=127.20.210.5
--webserver_interface=127.20.210.5
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:29.308137 5154 ts_manager.cc:194] Registered new tserver with Master: 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:29.308801 5154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.4:34433
W20240501 19:57:29.374322 5745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:29.374490 5745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:29.374553 5745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:29.377291 5745 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:29.377354 5745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:29.377451 5745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.5
I20240501 19:57:29.380677 5745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.5:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.20.210.5
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--never_fsync=true
--heap_profile_path=/tmp/kudu.5745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.5
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:29.381438 5745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:29.382412 5745 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:29.389583 5752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:29.390213 5754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:29.391021 5751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:29.391769 5745 server_base.cc:1034] running on GCE node
I20240501 19:57:29.392033 5745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:29.392434 5745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:29.393580 5745 hybrid_clock.cc:648] HybridClock initialized: now 1714593449393572 us; error 26 us; skew 500 ppm
I20240501 19:57:29.395129 5745 webserver.cc:460] Webserver started at http://127.20.210.5:40177/ using document root <none> and password file <none>
I20240501 19:57:29.395566 5745 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:29.395619 5745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:29.395771 5745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:29.397143 5745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "1865147cd5a346ad8549f18fa8cc26c3"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.397553 5745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "1865147cd5a346ad8549f18fa8cc26c3"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.400925 5745 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240501 19:57:29.402853 5760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.403741 5745 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:29.403873 5745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "1865147cd5a346ad8549f18fa8cc26c3"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.403951 5745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:29.443765 5745 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:29.444257 5745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:29.444407 5745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:29.444862 5745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:29.445667 5745 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:29.445712 5745 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.445763 5745 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:29.445816 5745 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.454059 5745 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.5:39499
I20240501 19:57:29.454093 5874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.5:39499 every 8 connection(s)
I20240501 19:57:29.456542 5745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240501 19:57:29.464113 5875 heartbeater.cc:347] Connected to a master server at 127.20.210.62:45969
I20240501 19:57:29.464334 5875 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:29.464761 5875 heartbeater.cc:511] Master 127.20.210.62:45969 requested a full tablet report, sending...
I20240501 19:57:29.465618 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 5745
I20240501 19:57:29.465725 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240501 19:57:29.469337 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.6:0
--local_ip_for_outbound_sockets=127.20.210.6
--webserver_interface=127.20.210.6
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:29.507383 5154 ts_manager.cc:194] Registered new tserver with Master: 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499)
I20240501 19:57:29.508055 5154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.5:57117
W20240501 19:57:29.577081 5879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:29.577242 5879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:29.577291 5879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:29.580114 5879 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:29.580170 5879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:29.580279 5879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.6
I20240501 19:57:29.583518 5879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.6:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.20.210.6
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--never_fsync=true
--heap_profile_path=/tmp/kudu.5879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.6
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:29.584247 5879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:29.585115 5879 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:29.592408 5888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:29.593132 5885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:29.594389 5886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:29.594430 5879 server_base.cc:1034] running on GCE node
I20240501 19:57:29.594729 5879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:29.595170 5879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:29.596313 5879 hybrid_clock.cc:648] HybridClock initialized: now 1714593449596296 us; error 31 us; skew 500 ppm
I20240501 19:57:29.597867 5879 webserver.cc:460] Webserver started at http://127.20.210.6:44995/ using document root <none> and password file <none>
I20240501 19:57:29.598361 5879 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:29.598419 5879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:29.598577 5879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:29.600045 5879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "9544b0e566c54a62a6135f3bff47c4f2"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.600502 5879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "9544b0e566c54a62a6135f3bff47c4f2"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.603368 5879 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 19:57:29.605373 5894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.606201 5879 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:29.606316 5879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "9544b0e566c54a62a6135f3bff47c4f2"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.606385 5879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:29.625763 5879 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:29.626215 5879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:29.626372 5879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:29.626813 5879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:29.627534 5879 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:29.627581 5879 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.627629 5879 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:29.627660 5879 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.635636 5879 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.6:34085
I20240501 19:57:29.635659 6009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.6:34085 every 8 connection(s)
I20240501 19:57:29.636379 5879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240501 19:57:29.643497 6010 heartbeater.cc:347] Connected to a master server at 127.20.210.62:45969
I20240501 19:57:29.643666 6010 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:29.644143 6010 heartbeater.cc:511] Master 127.20.210.62:45969 requested a full tablet report, sending...
I20240501 19:57:29.645031 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 5879
I20240501 19:57:29.645134 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240501 19:57:29.648404 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.7:0
--local_ip_for_outbound_sockets=127.20.210.7
--webserver_interface=127.20.210.7
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:29.685930 5154 ts_manager.cc:194] Registered new tserver with Master: 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085)
I20240501 19:57:29.686563 5154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.6:49863
W20240501 19:57:29.754863 6015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:29.755061 6015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:29.755110 6015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:29.758085 6015 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:29.758138 6015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:29.758194 6015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.7
I20240501 19:57:29.761344 6015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.7:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.20.210.7
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--never_fsync=true
--heap_profile_path=/tmp/kudu.6015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.7
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:29.761968 6015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:29.762852 6015 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:29.769964 6020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:29.770625 6021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:29.771378 6023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:29.772126 6015 server_base.cc:1034] running on GCE node
I20240501 19:57:29.772418 6015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:29.772866 6015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:29.774015 6015 hybrid_clock.cc:648] HybridClock initialized: now 1714593449773994 us; error 34 us; skew 500 ppm
I20240501 19:57:29.775701 6015 webserver.cc:460] Webserver started at http://127.20.210.7:39317/ using document root <none> and password file <none>
I20240501 19:57:29.776189 6015 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:29.776252 6015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:29.776415 6015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:29.777841 6015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "a881063bd86945a8871c7c102d8235c8"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.778296 6015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "a881063bd86945a8871c7c102d8235c8"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.781255 6015 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240501 19:57:29.783195 6029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.784188 6015 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:29.784278 6015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "a881063bd86945a8871c7c102d8235c8"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.784360 6015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:29.803423 6015 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:29.803927 6015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:29.804064 6015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:29.804538 6015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:29.805356 6015 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:29.805399 6015 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.805440 6015 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:29.805471 6015 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.810585 5343 heartbeater.cc:503] Master 127.20.210.62:45969 was elected leader, sending a full tablet report...
I20240501 19:57:29.813853 6015 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.7:39815
I20240501 19:57:29.813894 6143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.7:39815 every 8 connection(s)
I20240501 19:57:29.814661 6015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240501 19:57:29.823055 6144 heartbeater.cc:347] Connected to a master server at 127.20.210.62:45969
I20240501 19:57:29.823221 6144 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:29.823654 6144 heartbeater.cc:511] Master 127.20.210.62:45969 requested a full tablet report, sending...
I20240501 19:57:29.824201 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 6015
I20240501 19:57:29.824306 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240501 19:57:29.826033 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.8:0
--local_ip_for_outbound_sockets=127.20.210.8
--webserver_interface=127.20.210.8
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:29.890666 5154 ts_manager.cc:194] Registered new tserver with Master: a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:29.891357 5154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.7:35435
W20240501 19:57:29.932579 6147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:29.932744 6147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:29.932785 6147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:29.935631 6147 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:29.935681 6147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:29.935756 6147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.8
I20240501 19:57:29.938892 6147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.8:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.20.210.8
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--never_fsync=true
--heap_profile_path=/tmp/kudu.6147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.8
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:29.939575 6147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:29.940519 6147 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:29.947728 6154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:29.948314 6156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:29.949138 6153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:29.949934 6147 server_base.cc:1034] running on GCE node
I20240501 19:57:29.950285 6147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:29.950783 6147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:29.951936 6147 hybrid_clock.cc:648] HybridClock initialized: now 1714593449951922 us; error 25 us; skew 500 ppm
I20240501 19:57:29.953523 6147 webserver.cc:460] Webserver started at http://127.20.210.8:35027/ using document root <none> and password file <none>
I20240501 19:57:29.954030 6147 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:29.954089 6147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:29.954278 6147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:29.955726 6147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "363f4ac4964f41a6ba0f98d1f487db83"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.956169 6147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "363f4ac4964f41a6ba0f98d1f487db83"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.959106 6147 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20240501 19:57:29.961082 6162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.961962 6147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:29.962065 6147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "363f4ac4964f41a6ba0f98d1f487db83"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:29.962136 6147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:29.983062 5476 heartbeater.cc:503] Master 127.20.210.62:45969 was elected leader, sending a full tablet report...
I20240501 19:57:29.991644 6147 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:29.992120 6147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:29.992234 6147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:29.992640 6147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:29.993381 6147 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:29.993422 6147 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:29.993453 6147 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:29.993472 6147 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:30.000985 6147 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.8:41983
I20240501 19:57:30.001029 6276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.8:41983 every 8 connection(s)
I20240501 19:57:30.001725 6147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 19:57:30.009276 6277 heartbeater.cc:347] Connected to a master server at 127.20.210.62:45969
I20240501 19:57:30.009483 6277 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:30.009918 6277 heartbeater.cc:511] Master 127.20.210.62:45969 requested a full tablet report, sending...
I20240501 19:57:30.011024 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 6147
I20240501 19:57:30.011116 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240501 19:57:30.013844 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.9:0
--local_ip_for_outbound_sockets=127.20.210.9
--webserver_interface=127.20.210.9
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:30.053380 5154 ts_manager.cc:194] Registered new tserver with Master: 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983)
I20240501 19:57:30.053970 5154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.8:40467
W20240501 19:57:30.121253 6281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:30.121395 6281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:30.121429 6281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:30.124156 6281 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:30.124208 6281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:30.124290 6281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.9
I20240501 19:57:30.127319 6281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.9:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.20.210.9
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--never_fsync=true
--heap_profile_path=/tmp/kudu.6281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.9
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:30.127913 6281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:30.128749 6281 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240501 19:57:30.131145 5609 heartbeater.cc:503] Master 127.20.210.62:45969 was elected leader, sending a full tablet report...
W20240501 19:57:30.136193 6289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:30.136760 6286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:30.137321 6287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:30.138041 6281 server_base.cc:1034] running on GCE node
I20240501 19:57:30.138381 6281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:30.138840 6281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:30.139974 6281 hybrid_clock.cc:648] HybridClock initialized: now 1714593450139955 us; error 32 us; skew 500 ppm
I20240501 19:57:30.141498 6281 webserver.cc:460] Webserver started at http://127.20.210.9:37435/ using document root <none> and password file <none>
I20240501 19:57:30.141937 6281 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:30.141986 6281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:30.142136 6281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:30.143474 6281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/instance:
uuid: "143c54da1456472dadbf1d5d65e0a6f6"
format_stamp: "Formatted at 2024-05-01 19:57:30 on dist-test-slave-vgh4"
I20240501 19:57:30.143913 6281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal/instance:
uuid: "143c54da1456472dadbf1d5d65e0a6f6"
format_stamp: "Formatted at 2024-05-01 19:57:30 on dist-test-slave-vgh4"
I20240501 19:57:30.146735 6281 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240501 19:57:30.148521 6295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:30.149379 6281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:30.149508 6281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "143c54da1456472dadbf1d5d65e0a6f6"
format_stamp: "Formatted at 2024-05-01 19:57:30 on dist-test-slave-vgh4"
I20240501 19:57:30.149582 6281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:30.181346 6281 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:30.181830 6281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:30.181986 6281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:30.182457 6281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:30.183198 6281 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:30.183247 6281 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:30.183297 6281 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:30.183331 6281 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:30.191155 6281 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.9:45861
I20240501 19:57:30.191177 6409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.9:45861 every 8 connection(s)
I20240501 19:57:30.191884 6281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240501 19:57:30.199101 6410 heartbeater.cc:347] Connected to a master server at 127.20.210.62:45969
I20240501 19:57:30.199258 6410 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:30.199537 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 6281
I20240501 19:57:30.199653 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal/instance
I20240501 19:57:30.199882 6410 heartbeater.cc:511] Master 127.20.210.62:45969 requested a full tablet report, sending...
I20240501 19:57:30.201910 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.10:0
--local_ip_for_outbound_sockets=127.20.210.10
--webserver_interface=127.20.210.10
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:30.243229 5153 ts_manager.cc:194] Registered new tserver with Master: 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861)
I20240501 19:57:30.244145 5153 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.9:55881
W20240501 19:57:30.308734 6414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:30.308912 6414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:30.308987 6414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240501 19:57:30.310719 5742 heartbeater.cc:503] Master 127.20.210.62:45969 was elected leader, sending a full tablet report...
W20240501 19:57:30.312059 6414 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:30.312111 6414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:30.312167 6414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.10
I20240501 19:57:30.315147 6414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.10:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--webserver_interface=127.20.210.10
--webserver_port=0
--tserver_master_addrs=127.20.210.62:45969
--never_fsync=true
--heap_profile_path=/tmp/kudu.6414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.10
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:30.315796 6414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:30.316603 6414 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:30.323779 6420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:30.324301 6422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:30.325093 6419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:30.325706 6414 server_base.cc:1034] running on GCE node
I20240501 19:57:30.325945 6414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:30.326357 6414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:30.327500 6414 hybrid_clock.cc:648] HybridClock initialized: now 1714593450327487 us; error 24 us; skew 500 ppm
I20240501 19:57:30.328963 6414 webserver.cc:460] Webserver started at http://127.20.210.10:44709/ using document root <none> and password file <none>
I20240501 19:57:30.329376 6414 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:30.329469 6414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:30.329612 6414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240501 19:57:30.331064 6414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data/instance:
uuid: "16737daa1d234f35826143a1aa29dc0b"
format_stamp: "Formatted at 2024-05-01 19:57:30 on dist-test-slave-vgh4"
I20240501 19:57:30.331444 6414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal/instance:
uuid: "16737daa1d234f35826143a1aa29dc0b"
format_stamp: "Formatted at 2024-05-01 19:57:30 on dist-test-slave-vgh4"
I20240501 19:57:30.334141 6414 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240501 19:57:30.335968 6428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:30.336808 6414 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:30.336892 6414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal
uuid: "16737daa1d234f35826143a1aa29dc0b"
format_stamp: "Formatted at 2024-05-01 19:57:30 on dist-test-slave-vgh4"
I20240501 19:57:30.336959 6414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:30.356731 6414 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:30.357139 6414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:30.357259 6414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:30.357640 6414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:30.358327 6414 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:30.358371 6414 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:30.358417 6414 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:30.358443 6414 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:30.365681 6414 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.10:40603
I20240501 19:57:30.365720 6541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.10:40603 every 8 connection(s)
I20240501 19:57:30.366425 6414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data/info.pb
I20240501 19:57:30.367182 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 6414
I20240501 19:57:30.367314 21320 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal/instance
I20240501 19:57:30.373584 6542 heartbeater.cc:347] Connected to a master server at 127.20.210.62:45969
I20240501 19:57:30.373745 6542 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:30.374107 6542 heartbeater.cc:511] Master 127.20.210.62:45969 requested a full tablet report, sending...
I20240501 19:57:30.403039 5153 ts_manager.cc:194] Registered new tserver with Master: 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603)
I20240501 19:57:30.403676 5153 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.10:60501
I20240501 19:57:30.404402 21320 external_mini_cluster.cc:930] 10 TS(s) registered with all masters
I20240501 19:57:30.412648 21320 external_mini_cluster.cc:930] 10 TS(s) registered with all masters
I20240501 19:57:30.412709 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 6147
I20240501 19:57:30.418515 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 6281
I20240501 19:57:30.423779 21320 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu with pid 6414
I20240501 19:57:30.509964 5875 heartbeater.cc:503] Master 127.20.210.62:45969 was elected leader, sending a full tablet report...
I20240501 19:57:30.688524 6010 heartbeater.cc:503] Master 127.20.210.62:45969 was elected leader, sending a full tablet report...
I20240501 19:57:30.893463 6144 heartbeater.cc:503] Master 127.20.210.62:45969 was elected leader, sending a full tablet report...
I20240501 19:57:34.179841 5153 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47970:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 19:57:34.183058 5153 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 19:57:34.199429 5942 tablet_service.cc:1467] Processing CreateTablet for tablet 95e8db3b5d81468098c01c32c4d22e80 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.200357 5942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95e8db3b5d81468098c01c32c4d22e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.201179 5809 tablet_service.cc:1467] Processing CreateTablet for tablet 95e8db3b5d81468098c01c32c4d22e80 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.201925 5809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95e8db3b5d81468098c01c32c4d22e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.202494 5808 tablet_service.cc:1467] Processing CreateTablet for tablet 80fdea49202c4ab5b3972b2c9ad7cff5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.202494 5941 tablet_service.cc:1467] Processing CreateTablet for tablet 80fdea49202c4ab5b3972b2c9ad7cff5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.202702 5941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80fdea49202c4ab5b3972b2c9ad7cff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.202723 5808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80fdea49202c4ab5b3972b2c9ad7cff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.203279 5940 tablet_service.cc:1467] Processing CreateTablet for tablet bd8a8b5e92c74598a5399fe0ca8c048d (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.203485 5940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd8a8b5e92c74598a5399fe0ca8c048d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.203498 5542 tablet_service.cc:1467] Processing CreateTablet for tablet 80fdea49202c4ab5b3972b2c9ad7cff5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.204246 5542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80fdea49202c4ab5b3972b2c9ad7cff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.205716 5807 tablet_service.cc:1467] Processing CreateTablet for tablet bd8a8b5e92c74598a5399fe0ca8c048d (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.205941 5807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd8a8b5e92c74598a5399fe0ca8c048d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.208925 6570 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:34.209463 5541 tablet_service.cc:1467] Processing CreateTablet for tablet bd8a8b5e92c74598a5399fe0ca8c048d (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.209683 5541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd8a8b5e92c74598a5399fe0ca8c048d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.210002 5543 tablet_service.cc:1467] Processing CreateTablet for tablet 95e8db3b5d81468098c01c32c4d22e80 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.211159 6570 tablet_bootstrap.cc:654] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.212397 6570 log.cc:824] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:34.212296 6075 tablet_service.cc:1467] Processing CreateTablet for tablet bd8a8b5e92c74598a5399fe0ca8c048d (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.213128 6075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd8a8b5e92c74598a5399fe0ca8c048d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.212519 6076 tablet_service.cc:1467] Processing CreateTablet for tablet 80fdea49202c4ab5b3972b2c9ad7cff5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.214059 6076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80fdea49202c4ab5b3972b2c9ad7cff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.214438 5543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95e8db3b5d81468098c01c32c4d22e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.215235 5410 tablet_service.cc:1467] Processing CreateTablet for tablet 95e8db3b5d81468098c01c32c4d22e80 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.215966 5410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95e8db3b5d81468098c01c32c4d22e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.212302 6077 tablet_service.cc:1467] Processing CreateTablet for tablet 95e8db3b5d81468098c01c32c4d22e80 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.216281 6077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95e8db3b5d81468098c01c32c4d22e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.216437 5409 tablet_service.cc:1467] Processing CreateTablet for tablet bd8a8b5e92c74598a5399fe0ca8c048d (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.216670 5409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd8a8b5e92c74598a5399fe0ca8c048d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.217003 5408 tablet_service.cc:1467] Processing CreateTablet for tablet 80fdea49202c4ab5b3972b2c9ad7cff5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.217214 5408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80fdea49202c4ab5b3972b2c9ad7cff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.217999 6570 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:34.220461 5675 tablet_service.cc:1467] Processing CreateTablet for tablet 95e8db3b5d81468098c01c32c4d22e80 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.221197 5675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95e8db3b5d81468098c01c32c4d22e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.221849 5277 tablet_service.cc:1467] Processing CreateTablet for tablet 95e8db3b5d81468098c01c32c4d22e80 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.222537 5674 tablet_service.cc:1467] Processing CreateTablet for tablet 80fdea49202c4ab5b3972b2c9ad7cff5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.222620 5277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95e8db3b5d81468098c01c32c4d22e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.222772 5674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80fdea49202c4ab5b3972b2c9ad7cff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.223008 5673 tablet_service.cc:1467] Processing CreateTablet for tablet bd8a8b5e92c74598a5399fe0ca8c048d (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.223212 5673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd8a8b5e92c74598a5399fe0ca8c048d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.223659 6570 ts_tablet_manager.cc:1367] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20240501 19:57:34.225741 5276 tablet_service.cc:1467] Processing CreateTablet for tablet 80fdea49202c4ab5b3972b2c9ad7cff5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.225955 5276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80fdea49202c4ab5b3972b2c9ad7cff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.226687 6570 raft_consensus.cc:348] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.226923 6570 raft_consensus.cc:374] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.226969 6570 raft_consensus.cc:729] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.227357 6570 consensus_queue.cc:260] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.228192 6570 ts_tablet_manager.cc:1398] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240501 19:57:34.229339 6570 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:34.230338 6570 tablet_bootstrap.cc:654] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.231101 6577 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:34.231216 6572 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:34.231734 6571 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:34.232242 5275 tablet_service.cc:1467] Processing CreateTablet for tablet bd8a8b5e92c74598a5399fe0ca8c048d (DEFAULT_TABLE table=rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.232507 5275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd8a8b5e92c74598a5399fe0ca8c048d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.233043 6577 tablet_bootstrap.cc:654] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.234133 6577 log.cc:824] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:34.234695 6574 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:34.235000 6572 tablet_bootstrap.cc:654] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.235392 6575 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:34.235719 6572 log.cc:824] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:34.236382 6574 tablet_bootstrap.cc:654] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.236531 6571 tablet_bootstrap.cc:654] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.237103 6575 tablet_bootstrap.cc:654] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.237083 6574 log.cc:824] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:34.237282 6571 log.cc:824] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:34.237648 6579 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:34.237819 6575 log.cc:824] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:34.238688 6576 raft_consensus.cc:482] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:34.238817 6571 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:34.238986 6571 ts_tablet_manager.cc:1367] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20240501 19:57:34.238922 6576 raft_consensus.cc:504] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.239197 6570 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:34.239423 6579 tablet_bootstrap.cc:654] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.240141 6579 log.cc:824] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:34.241067 6576 leader_election.cc:288] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
I20240501 19:57:34.241881 6579 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:34.242029 6579 ts_tablet_manager.cc:1367] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240501 19:57:34.242823 6572 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:34.242769 6571 raft_consensus.cc:348] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.242972 6572 ts_tablet_manager.cc:1367] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20240501 19:57:34.242991 6571 raft_consensus.cc:374] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.243052 6571 raft_consensus.cc:729] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.243276 6570 ts_tablet_manager.cc:1367] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20240501 19:57:34.243278 6574 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:34.243420 6574 ts_tablet_manager.cc:1367] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20240501 19:57:34.243451 6571 consensus_queue.cc:260] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.243789 6570 raft_consensus.cc:348] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.243906 6570 raft_consensus.cc:374] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.243933 6570 raft_consensus.cc:729] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.244066 6570 consensus_queue.cc:260] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.244273 6571 ts_tablet_manager.cc:1398] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20240501 19:57:34.244323 6570 ts_tablet_manager.cc:1398] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:34.244570 6571 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:34.245445 6571 tablet_bootstrap.cc:654] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.246179 6570 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:34.246289 6572 raft_consensus.cc:348] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.246490 6572 raft_consensus.cc:374] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.246533 6572 raft_consensus.cc:729] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.246878 6572 consensus_queue.cc:260] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.247152 6577 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:34.247279 6577 ts_tablet_manager.cc:1367] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20240501 19:57:34.247351 6570 tablet_bootstrap.cc:654] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.247609 6572 ts_tablet_manager.cc:1398] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240501 19:57:34.248378 6572 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:34.249181 6572 tablet_bootstrap.cc:654] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.249778 6575 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:34.249704 6577 raft_consensus.cc:348] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.249914 6575 ts_tablet_manager.cc:1367] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20240501 19:57:34.249945 6577 raft_consensus.cc:374] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.249985 6577 raft_consensus.cc:729] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.250308 6572 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:34.250381 6572 ts_tablet_manager.cc:1367] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:34.250247 6577 consensus_queue.cc:260] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.251065 6577 ts_tablet_manager.cc:1398] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240501 19:57:34.252403 6577 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:34.253229 6577 tablet_bootstrap.cc:654] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.253325 6575 raft_consensus.cc:348] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.253587 6575 raft_consensus.cc:374] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.253675 6575 raft_consensus.cc:729] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.254029 6572 raft_consensus.cc:348] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.254048 6575 consensus_queue.cc:260] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.254179 6572 raft_consensus.cc:374] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.254283 6572 raft_consensus.cc:729] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.254446 6572 consensus_queue.cc:260] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.254786 6577 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:34.254818 6575 ts_tablet_manager.cc:1398] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240501 19:57:34.254870 6577 ts_tablet_manager.cc:1367] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:34.255122 6575 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:34.255303 6577 raft_consensus.cc:348] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.255401 6577 raft_consensus.cc:374] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.255429 6577 raft_consensus.cc:729] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.255532 6577 consensus_queue.cc:260] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.254851 6574 raft_consensus.cc:348] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.255872 6577 ts_tablet_manager.cc:1398] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.255909 6574 raft_consensus.cc:374] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.255959 6574 raft_consensus.cc:729] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.256011 6575 tablet_bootstrap.cc:654] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.256031 6577 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:34.256336 6574 consensus_queue.cc:260] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.256716 6572 ts_tablet_manager.cc:1398] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240501 19:57:34.256846 6577 tablet_bootstrap.cc:654] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.256878 6572 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:34.257086 6574 ts_tablet_manager.cc:1398] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20240501 19:57:34.257408 6574 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:34.257686 6572 tablet_bootstrap.cc:654] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
W20240501 19:57:34.258548 5344 tablet.cc:2369] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:57:34.258314 6579 raft_consensus.cc:348] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.259074 6579 raft_consensus.cc:374] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.259136 6577 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:34.259135 6579 raft_consensus.cc:729] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.259200 6577 ts_tablet_manager.cc:1367] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:34.259490 6579 consensus_queue.cc:260] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.259552 6577 raft_consensus.cc:348] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.259642 6577 raft_consensus.cc:374] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.259670 6577 raft_consensus.cc:729] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.259820 6577 consensus_queue.cc:260] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.260071 6577 ts_tablet_manager.cc:1398] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.260260 6579 ts_tablet_manager.cc:1398] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20240501 19:57:34.260562 6579 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:34.260571 6574 tablet_bootstrap.cc:654] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.261366 6579 tablet_bootstrap.cc:654] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.263675 6571 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:34.263830 6571 ts_tablet_manager.cc:1367] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.001s
I20240501 19:57:34.263931 6574 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:34.264003 6574 ts_tablet_manager.cc:1367] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240501 19:57:34.264379 6571 raft_consensus.cc:348] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.264510 6571 raft_consensus.cc:374] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.264541 6571 raft_consensus.cc:729] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.264509 6574 raft_consensus.cc:348] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.264611 6574 raft_consensus.cc:374] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.264647 6574 raft_consensus.cc:729] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.264685 6571 consensus_queue.cc:260] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.264751 6574 consensus_queue.cc:260] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.264969 6571 ts_tablet_manager.cc:1398] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.265033 6574 ts_tablet_manager.cc:1398] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:34.265139 6571 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:34.265197 6574 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:34.265950 6571 tablet_bootstrap.cc:654] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.265995 6574 tablet_bootstrap.cc:654] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.267784 6570 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:34.267832 6575 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:34.267868 6570 ts_tablet_manager.cc:1367] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.001s
I20240501 19:57:34.267905 6575 ts_tablet_manager.cc:1367] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20240501 19:57:34.268287 6570 raft_consensus.cc:348] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.268388 6570 raft_consensus.cc:374] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.268419 6570 raft_consensus.cc:729] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.268393 6575 raft_consensus.cc:348] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.268497 6575 raft_consensus.cc:374] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.268536 6575 raft_consensus.cc:729] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.268522 6570 consensus_queue.cc:260] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.268642 6575 consensus_queue.cc:260] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.268829 6570 ts_tablet_manager.cc:1398] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:34.268936 6575 ts_tablet_manager.cc:1398] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:34.269081 6575 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:34.269258 6574 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:34.269332 6574 ts_tablet_manager.cc:1367] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240501 19:57:34.269716 6574 raft_consensus.cc:348] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.269902 6574 raft_consensus.cc:374] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.269919 6575 tablet_bootstrap.cc:654] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.269940 6574 raft_consensus.cc:729] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.270061 6574 consensus_queue.cc:260] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.271077 6572 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:34.271143 6572 ts_tablet_manager.cc:1367] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240501 19:57:34.271490 6572 raft_consensus.cc:348] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.271580 6572 raft_consensus.cc:374] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.271603 6572 raft_consensus.cc:729] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.271690 6572 consensus_queue.cc:260] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.271907 6572 ts_tablet_manager.cc:1398] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.274173 5297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80fdea49202c4ab5b3972b2c9ad7cff5" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:34.274431 5297 raft_consensus.cc:2454] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:34.274753 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80fdea49202c4ab5b3972b2c9ad7cff5" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:34.274953 5430 raft_consensus.cc:2454] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
W20240501 19:57:34.274987 5743 tablet.cc:2369] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:57:34.276456 6574 ts_tablet_manager.cc:1398] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.007s user 0.000s sys 0.001s
I20240501 19:57:34.277300 5829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80fdea49202c4ab5b3972b2c9ad7cff5" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:34.277506 5829 raft_consensus.cc:2454] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:34.277885 5497 leader_election.cc:302] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:34.278090 6576 raft_consensus.cc:2781] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:34.278168 6576 raft_consensus.cc:482] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:34.278209 6576 raft_consensus.cc:3037] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.279309 6576 raft_consensus.cc:504] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.279959 6576 leader_election.cc:288] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Requested vote from peers 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
I20240501 19:57:34.280133 5829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80fdea49202c4ab5b3972b2c9ad7cff5" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:34.280195 5297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80fdea49202c4ab5b3972b2c9ad7cff5" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:34.280230 5829 raft_consensus.cc:3037] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.280308 5297 raft_consensus.cc:3037] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.281098 5297 raft_consensus.cc:2454] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:34.281395 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80fdea49202c4ab5b3972b2c9ad7cff5" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:34.281481 5430 raft_consensus.cc:3037] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.281638 5829 raft_consensus.cc:2454] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:34.282286 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80fdea49202c4ab5b3972b2c9ad7cff5" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:34.282325 5961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80fdea49202c4ab5b3972b2c9ad7cff5" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:34.282413 5430 raft_consensus.cc:2454] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:34.282498 5962 raft_consensus.cc:2454] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:34.282961 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80fdea49202c4ab5b3972b2c9ad7cff5" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:34.283051 6579 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:34.282972 6095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80fdea49202c4ab5b3972b2c9ad7cff5" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:34.283111 6579 ts_tablet_manager.cc:1367] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20240501 19:57:34.283170 6097 raft_consensus.cc:2454] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:34.283509 5497 leader_election.cc:302] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0, f5287d5ae4d84bd692b0f7985cd94171; no voters: 9544b0e566c54a62a6135f3bff47c4f2, a881063bd86945a8871c7c102d8235c8
I20240501 19:57:34.283584 6579 raft_consensus.cc:348] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.283663 6576 raft_consensus.cc:2781] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:34.283689 6579 raft_consensus.cc:374] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.283797 6576 raft_consensus.cc:686] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Becoming Leader. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Running, Role: LEADER
I20240501 19:57:34.283739 6579 raft_consensus.cc:729] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.283968 6579 consensus_queue.cc:260] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.284067 6576 consensus_queue.cc:237] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.284217 6579 ts_tablet_manager.cc:1398] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.284376 6579 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:34.285234 6579 tablet_bootstrap.cc:654] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.285405 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80fdea49202c4ab5b3972b2c9ad7cff5" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:34.285642 5696 raft_consensus.cc:2454] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:34.285405 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80fdea49202c4ab5b3972b2c9ad7cff5" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:34.285845 5695 raft_consensus.cc:3037] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.286734 5695 raft_consensus.cc:2454] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:34.286828 6575 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:34.286898 6575 ts_tablet_manager.cc:1367] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20240501 19:57:34.287247 6575 raft_consensus.cc:348] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.287356 6575 raft_consensus.cc:374] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.287386 6575 raft_consensus.cc:729] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.287498 6575 consensus_queue.cc:260] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.287735 6575 ts_tablet_manager.cc:1398] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.288441 6571 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:34.288518 6571 ts_tablet_manager.cc:1367] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.001s
I20240501 19:57:34.288655 5153 catalog_manager.cc:5484] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:34.289036 6571 raft_consensus.cc:348] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.289150 6571 raft_consensus.cc:374] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.289180 6571 raft_consensus.cc:729] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.289305 6571 consensus_queue.cc:260] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.289573 6571 ts_tablet_manager.cc:1398] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:34.290056 6579 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:34.290127 6579 ts_tablet_manager.cc:1367] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 19:57:34.291016 6579 raft_consensus.cc:348] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.291467 6579 raft_consensus.cc:374] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.291518 6579 raft_consensus.cc:729] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.291633 6579 consensus_queue.cc:260] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.291908 6579 ts_tablet_manager.cc:1398] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:34.313278 6600 raft_consensus.cc:482] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:34.313365 6600 raft_consensus.cc:504] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.315582 6600 leader_election.cc:288] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
W20240501 19:57:34.318076 6145 tablet.cc:2369] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:57:34.318722 5961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8a8b5e92c74598a5399fe0ca8c048d" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:34.318830 5961 raft_consensus.cc:2454] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:34.318861 5829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8a8b5e92c74598a5399fe0ca8c048d" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:34.318962 5829 raft_consensus.cc:2454] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:34.319438 5297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8a8b5e92c74598a5399fe0ca8c048d" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:34.319561 5297 raft_consensus.cc:2454] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:34.319885 5628 leader_election.cc:302] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:34.320111 6600 raft_consensus.cc:2781] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:34.320168 6600 raft_consensus.cc:482] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:34.320209 6600 raft_consensus.cc:3037] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.321305 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8a8b5e92c74598a5399fe0ca8c048d" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:34.321429 6097 raft_consensus.cc:2454] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:34.321416 6600 raft_consensus.cc:504] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.321965 6600 leader_election.cc:288] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 election: Requested vote from peers 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:34.322225 5829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8a8b5e92c74598a5399fe0ca8c048d" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:34.322363 5829 raft_consensus.cc:3037] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.322468 5961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8a8b5e92c74598a5399fe0ca8c048d" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:34.322530 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8a8b5e92c74598a5399fe0ca8c048d" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:34.322564 5961 raft_consensus.cc:3037] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.322616 6097 raft_consensus.cc:3037] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.323113 5297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8a8b5e92c74598a5399fe0ca8c048d" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:34.323215 5297 raft_consensus.cc:3037] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.323305 5829 raft_consensus.cc:2454] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:34.323540 5961 raft_consensus.cc:2454] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:34.323540 6097 raft_consensus.cc:2454] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:34.323925 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8a8b5e92c74598a5399fe0ca8c048d" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:34.324043 5430 raft_consensus.cc:2454] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:34.324035 5631 leader_election.cc:302] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, a881063bd86945a8871c7c102d8235c8; no voters:
I20240501 19:57:34.324123 5297 raft_consensus.cc:2454] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:34.324242 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8a8b5e92c74598a5399fe0ca8c048d" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:34.324285 5429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8a8b5e92c74598a5399fe0ca8c048d" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:34.324391 5429 raft_consensus.cc:3037] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.324417 5562 raft_consensus.cc:3037] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.324429 6600 raft_consensus.cc:2781] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:34.324477 5563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8a8b5e92c74598a5399fe0ca8c048d" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:34.324600 6600 raft_consensus.cc:686] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Becoming Leader. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Running, Role: LEADER
I20240501 19:57:34.324874 6600 consensus_queue.cc:237] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.325311 5429 raft_consensus.cc:2454] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:34.325358 5562 raft_consensus.cc:2454] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:34.327719 5153 catalog_manager.cc:5484] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d4855c096ac4e1e936abf15834332ae (127.20.210.4). New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } }
W20240501 19:57:34.342574 5610 tablet.cc:2369] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 19:57:34.390097 6012 tablet.cc:2369] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240501 19:57:34.414932 6598 raft_consensus.cc:482] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:34.415033 6598 raft_consensus.cc:504] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.416072 6598 leader_election.cc:288] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
I20240501 19:57:34.419915 5961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e8db3b5d81468098c01c32c4d22e80" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:34.420030 5961 raft_consensus.cc:2454] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:34.421566 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e8db3b5d81468098c01c32c4d22e80" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:34.421675 6097 raft_consensus.cc:2454] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:34.422263 5429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e8db3b5d81468098c01c32c4d22e80" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:34.422390 5429 raft_consensus.cc:2454] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:34.422598 5829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e8db3b5d81468098c01c32c4d22e80" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:34.422713 5829 raft_consensus.cc:2454] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:34.422770 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e8db3b5d81468098c01c32c4d22e80" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:34.422889 5695 raft_consensus.cc:2454] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:34.423293 5230 leader_election.cc:302] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 9544b0e566c54a62a6135f3bff47c4f2, 9782508a47ba4403a39ca49fff7677fa, a881063bd86945a8871c7c102d8235c8, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:34.423544 6598 raft_consensus.cc:2781] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:34.423612 6598 raft_consensus.cc:482] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:34.423648 6598 raft_consensus.cc:3037] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.423990 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e8db3b5d81468098c01c32c4d22e80" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:34.424124 5562 raft_consensus.cc:2454] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:34.424537 6598 raft_consensus.cc:504] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.424984 6598 leader_election.cc:288] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [CANDIDATE]: Term 1 election: Requested vote from peers 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
I20240501 19:57:34.425191 5829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e8db3b5d81468098c01c32c4d22e80" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:34.425176 5961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e8db3b5d81468098c01c32c4d22e80" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:34.425261 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e8db3b5d81468098c01c32c4d22e80" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:34.425292 5961 raft_consensus.cc:3037] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.425293 5829 raft_consensus.cc:3037] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.425284 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e8db3b5d81468098c01c32c4d22e80" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:34.425366 6097 raft_consensus.cc:3037] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.425386 5695 raft_consensus.cc:3037] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.426195 5961 raft_consensus.cc:2454] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:34.426195 5695 raft_consensus.cc:2454] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:34.426225 5829 raft_consensus.cc:2454] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:34.426225 6097 raft_consensus.cc:2454] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:34.426596 5233 leader_election.cc:302] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 9544b0e566c54a62a6135f3bff47c4f2, a881063bd86945a8871c7c102d8235c8, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:34.426661 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e8db3b5d81468098c01c32c4d22e80" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:34.426784 5562 raft_consensus.cc:3037] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.426959 6598 raft_consensus.cc:2781] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:34.426997 5429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e8db3b5d81468098c01c32c4d22e80" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:34.427042 6598 raft_consensus.cc:686] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Becoming Leader. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Running, Role: LEADER
I20240501 19:57:34.427088 5429 raft_consensus.cc:3037] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.427273 6598 consensus_queue.cc:237] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:34.427714 5562 raft_consensus.cc:2454] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:34.427973 5429 raft_consensus.cc:2454] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:34.430055 5153 catalog_manager.cc:5484] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } }
W20240501 19:57:34.440537 5477 tablet.cc:2369] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 19:57:34.462848 5876 tablet.cc:2369] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240501 19:57:34.580718 6629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:34.580914 6629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:34.583554 6629 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240501 19:57:34.656244 5429 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.656931 5829 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.656985 5562 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.657493 5961 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.657557 6097 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.658041 5695 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.659443 5829 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.659818 6665 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.660044 6097 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.660059 5297 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.660315 6665 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.660588 6665 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.660892 6600 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.660975 6665 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.661197 6665 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.661438 6665 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.668968 5428 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.669001 6095 raft_consensus.cc:3037] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.669832 5827 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.670179 6095 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.670408 6581 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.670732 6576 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.670833 6581 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.671519 5695 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.672401 6620 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.672550 6576 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.672701 6620 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.673409 5960 raft_consensus.cc:3037] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.674185 5296 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.674450 5960 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.674533 5562 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.674855 5961 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.675299 6576 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.677014 6600 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.677531 6600 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.678226 5429 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:34.678375 6576 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.679142 6620 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:34.702116 6659 mvcc.cc:205] Tried to move back new op lower bound from 7022974790383869952 to 7022974788751790080. Current Snapshot: MvccSnapshot[applied={T|T < 7022974790317993984}]
I20240501 19:57:34.706292 6702 mvcc.cc:205] Tried to move back new op lower bound from 7022974790317993984 to 7022974788751790080. Current Snapshot: MvccSnapshot[applied={T|T < 7022974790317993984}]
I20240501 19:57:34.715237 6690 mvcc.cc:205] Tried to move back new op lower bound from 7022974790278090752 to 7022974788918575104. Current Snapshot: MvccSnapshot[applied={T|T < 7022974790278090752}]
I20240501 19:57:34.717278 6695 mvcc.cc:205] Tried to move back new op lower bound from 7022974790278090752 to 7022974788918575104. Current Snapshot: MvccSnapshot[applied={T|T < 7022974790278090752}]
I20240501 19:57:34.719404 6680 mvcc.cc:205] Tried to move back new op lower bound from 7022974790317993984 to 7022974788751790080. Current Snapshot: MvccSnapshot[applied={T|T < 7022974790317993984}]
I20240501 19:57:34.722467 6692 mvcc.cc:205] Tried to move back new op lower bound from 7022974790264479744 to 7022974789337890816. Current Snapshot: MvccSnapshot[applied={T|T < 7022974790264479744}]
I20240501 19:57:34.728192 6661 mvcc.cc:205] Tried to move back new op lower bound from 7022974790264479744 to 7022974789337890816. Current Snapshot: MvccSnapshot[applied={T|T < 7022974790264479744}]
INSERT report
rows total: 2000
time total: 189.77 ms
time per row: 0.0948851 ms
I20240501 19:57:34.814843 5147 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47970:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 19:57:34.815338 5147 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_1 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 19:57:34.820437 5807 tablet_service.cc:1467] Processing CreateTablet for tablet 85d29eb6552945a7879ee1a7e12919f0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.820739 5807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d29eb6552945a7879ee1a7e12919f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.822785 5942 tablet_service.cc:1467] Processing CreateTablet for tablet 85d29eb6552945a7879ee1a7e12919f0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.822892 6075 tablet_service.cc:1467] Processing CreateTablet for tablet 85d29eb6552945a7879ee1a7e12919f0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.822916 5543 tablet_service.cc:1467] Processing CreateTablet for tablet 0681575c2bb142e0915be3dc75858f87 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.823025 5942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d29eb6552945a7879ee1a7e12919f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.823143 5543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0681575c2bb142e0915be3dc75858f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.823153 6075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d29eb6552945a7879ee1a7e12919f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.824187 5410 tablet_service.cc:1467] Processing CreateTablet for tablet e17a6ae467bf42abac714a94e1ec28e5 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.824431 5410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e17a6ae467bf42abac714a94e1ec28e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.825349 5409 tablet_service.cc:1467] Processing CreateTablet for tablet 85d29eb6552945a7879ee1a7e12919f0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.825412 5808 tablet_service.cc:1467] Processing CreateTablet for tablet 0681575c2bb142e0915be3dc75858f87 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.825549 5409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d29eb6552945a7879ee1a7e12919f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.825502 6077 tablet_service.cc:1467] Processing CreateTablet for tablet 0681575c2bb142e0915be3dc75858f87 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.825613 5808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0681575c2bb142e0915be3dc75858f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.825690 6077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0681575c2bb142e0915be3dc75858f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.826509 5809 tablet_service.cc:1467] Processing CreateTablet for tablet e17a6ae467bf42abac714a94e1ec28e5 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.826686 5809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e17a6ae467bf42abac714a94e1ec28e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.827380 5408 tablet_service.cc:1467] Processing CreateTablet for tablet 0681575c2bb142e0915be3dc75858f87 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.827560 5408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0681575c2bb142e0915be3dc75858f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.827641 5543 tablet_service.cc:1467] Processing CreateTablet for tablet e17a6ae467bf42abac714a94e1ec28e5 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.827826 5543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e17a6ae467bf42abac714a94e1ec28e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.828080 6739 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:34.828215 5542 tablet_service.cc:1467] Processing CreateTablet for tablet 85d29eb6552945a7879ee1a7e12919f0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.828429 5542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d29eb6552945a7879ee1a7e12919f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.828866 6739 tablet_bootstrap.cc:654] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.829770 6075 tablet_service.cc:1467] Processing CreateTablet for tablet e17a6ae467bf42abac714a94e1ec28e5 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.829784 5941 tablet_service.cc:1467] Processing CreateTablet for tablet e17a6ae467bf42abac714a94e1ec28e5 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.830009 6075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e17a6ae467bf42abac714a94e1ec28e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.830009 5941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e17a6ae467bf42abac714a94e1ec28e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.830283 5675 tablet_service.cc:1467] Processing CreateTablet for tablet 85d29eb6552945a7879ee1a7e12919f0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.830540 5675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d29eb6552945a7879ee1a7e12919f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.830685 5277 tablet_service.cc:1467] Processing CreateTablet for tablet 85d29eb6552945a7879ee1a7e12919f0 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.830780 5674 tablet_service.cc:1467] Processing CreateTablet for tablet e17a6ae467bf42abac714a94e1ec28e5 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.830945 5277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d29eb6552945a7879ee1a7e12919f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.830976 5674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e17a6ae467bf42abac714a94e1ec28e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.832485 5942 tablet_service.cc:1467] Processing CreateTablet for tablet 0681575c2bb142e0915be3dc75858f87 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.832700 5942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0681575c2bb142e0915be3dc75858f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.832876 5673 tablet_service.cc:1467] Processing CreateTablet for tablet 0681575c2bb142e0915be3dc75858f87 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.833063 5673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0681575c2bb142e0915be3dc75858f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.833096 5276 tablet_service.cc:1467] Processing CreateTablet for tablet 0681575c2bb142e0915be3dc75858f87 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.833312 5276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0681575c2bb142e0915be3dc75858f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.835218 5275 tablet_service.cc:1467] Processing CreateTablet for tablet e17a6ae467bf42abac714a94e1ec28e5 (DEFAULT_TABLE table=rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:34.835435 5275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e17a6ae467bf42abac714a94e1ec28e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:34.836350 6735 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:34.837057 6735 tablet_bootstrap.cc:654] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.837742 6738 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:34.837978 6739 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:34.837992 6735 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:34.838063 6739 ts_tablet_manager.cc:1367] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240501 19:57:34.838132 6735 ts_tablet_manager.cc:1367] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:34.838546 6738 tablet_bootstrap.cc:654] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.838531 6739 raft_consensus.cc:348] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.838536 6735 raft_consensus.cc:348] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.838652 6739 raft_consensus.cc:374] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.838685 6735 raft_consensus.cc:374] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.838696 6739 raft_consensus.cc:729] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.838748 6735 raft_consensus.cc:729] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.838846 6739 consensus_queue.cc:260] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.838896 6735 consensus_queue.cc:260] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.839135 6739 ts_tablet_manager.cc:1398] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.839254 6735 ts_tablet_manager.cc:1398] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:34.839277 6739 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:34.839399 6735 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:34.840108 6739 tablet_bootstrap.cc:654] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.840863 6735 tablet_bootstrap.cc:654] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.841158 6739 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:34.841235 6739 ts_tablet_manager.cc:1367] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:34.841619 6739 raft_consensus.cc:348] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.841730 6739 raft_consensus.cc:374] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.841766 6739 raft_consensus.cc:729] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.841869 6739 consensus_queue.cc:260] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.842128 6739 ts_tablet_manager.cc:1398] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.842288 6739 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:34.843148 6739 tablet_bootstrap.cc:654] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.844295 6739 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:34.844363 6739 ts_tablet_manager.cc:1367] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:34.844729 6739 raft_consensus.cc:348] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.844838 6739 raft_consensus.cc:374] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.844882 6739 raft_consensus.cc:729] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.845010 6739 consensus_queue.cc:260] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.845250 6739 ts_tablet_manager.cc:1398] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.846411 6736 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:34.847127 6736 tablet_bootstrap.cc:654] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.847628 6737 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:34.847930 6736 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:34.848008 6736 ts_tablet_manager.cc:1367] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:34.848301 6737 tablet_bootstrap.cc:654] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.848299 6736 raft_consensus.cc:348] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.848416 6736 raft_consensus.cc:374] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.848450 6736 raft_consensus.cc:729] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.848568 6736 consensus_queue.cc:260] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.848778 6742 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:34.848904 6736 ts_tablet_manager.cc:1398] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.849079 6736 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:34.849540 6742 tablet_bootstrap.cc:654] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.849782 6736 tablet_bootstrap.cc:654] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.850466 6735 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:34.850536 6735 ts_tablet_manager.cc:1367] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240501 19:57:34.850875 6735 raft_consensus.cc:348] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.850991 6735 raft_consensus.cc:374] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.851037 6735 raft_consensus.cc:729] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.851209 6737 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:34.851172 6735 consensus_queue.cc:260] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.851363 6737 ts_tablet_manager.cc:1367] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 19:57:34.851835 6735 ts_tablet_manager.cc:1398] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.851783 6737 raft_consensus.cc:348] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.851945 6737 raft_consensus.cc:374] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.851974 6735 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:34.852032 6737 raft_consensus.cc:729] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.852210 6737 consensus_queue.cc:260] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.852542 6737 ts_tablet_manager.cc:1398] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.852669 6737 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:34.853576 6737 tablet_bootstrap.cc:654] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.854516 6735 tablet_bootstrap.cc:654] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.854578 6741 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:34.854707 6742 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:34.854792 6742 ts_tablet_manager.cc:1367] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 19:57:34.855098 6742 raft_consensus.cc:348] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.855185 6742 raft_consensus.cc:374] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.855211 6742 raft_consensus.cc:729] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.855335 6741 tablet_bootstrap.cc:654] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.855355 6742 consensus_queue.cc:260] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.855758 6735 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:34.855868 6735 ts_tablet_manager.cc:1367] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 19:57:34.855621 6742 ts_tablet_manager.cc:1398] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.856204 6742 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:34.856202 6738 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:34.856287 6738 ts_tablet_manager.cc:1367] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20240501 19:57:34.856674 6738 raft_consensus.cc:348] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.856814 6738 raft_consensus.cc:374] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.856853 6738 raft_consensus.cc:729] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.856985 6738 consensus_queue.cc:260] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.857153 6742 tablet_bootstrap.cc:654] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.857056 6735 raft_consensus.cc:348] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.857254 6738 ts_tablet_manager.cc:1398] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:34.857276 6735 raft_consensus.cc:374] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.857381 6735 raft_consensus.cc:729] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.857434 6738 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:34.857583 6735 consensus_queue.cc:260] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.858103 6737 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:34.858206 6737 ts_tablet_manager.cc:1367] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240501 19:57:34.858237 6738 tablet_bootstrap.cc:654] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.858460 6735 ts_tablet_manager.cc:1398] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:57:34.858609 6737 raft_consensus.cc:348] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.858763 6737 raft_consensus.cc:374] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.858996 6737 raft_consensus.cc:729] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.859150 6737 consensus_queue.cc:260] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.859292 6738 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:34.859362 6738 ts_tablet_manager.cc:1367] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:57:34.859800 6738 raft_consensus.cc:348] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.859966 6742 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:34.859944 6738 raft_consensus.cc:374] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.860020 6738 raft_consensus.cc:729] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.860044 6742 ts_tablet_manager.cc:1367] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:57:34.860179 6738 consensus_queue.cc:260] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.860463 6742 raft_consensus.cc:348] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.860564 6742 raft_consensus.cc:374] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.860597 6742 raft_consensus.cc:729] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.860863 6737 ts_tablet_manager.cc:1398] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240501 19:57:34.860895 6738 ts_tablet_manager.cc:1398] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:34.861039 6738 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:34.861054 6737 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:34.861841 6738 tablet_bootstrap.cc:654] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.861869 6737 tablet_bootstrap.cc:654] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.862371 6742 consensus_queue.cc:260] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.862679 6742 ts_tablet_manager.cc:1398] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:57:34.862833 6742 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:34.863631 6742 tablet_bootstrap.cc:654] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.864614 6742 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:34.864672 6736 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:34.864707 6742 ts_tablet_manager.cc:1367] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:34.864768 6736 ts_tablet_manager.cc:1367] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240501 19:57:34.865100 6742 raft_consensus.cc:348] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.865200 6742 raft_consensus.cc:374] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.865131 6736 raft_consensus.cc:348] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.865221 6742 raft_consensus.cc:729] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.865240 6736 raft_consensus.cc:374] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.865284 6736 raft_consensus.cc:729] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.865312 6742 consensus_queue.cc:260] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.865402 6736 consensus_queue.cc:260] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.865559 6742 ts_tablet_manager.cc:1398] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:34.865658 6736 ts_tablet_manager.cc:1398] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:34.865803 6736 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:34.866806 6736 tablet_bootstrap.cc:654] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.866853 6741 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:34.866961 6741 ts_tablet_manager.cc:1367] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240501 19:57:34.867329 6741 raft_consensus.cc:348] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.867426 6741 raft_consensus.cc:374] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.867452 6741 raft_consensus.cc:729] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.867548 6741 consensus_queue.cc:260] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.867682 6736 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:34.867751 6736 ts_tablet_manager.cc:1367] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 19:57:34.867790 6741 ts_tablet_manager.cc:1398] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:34.867914 6741 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:34.868149 6736 raft_consensus.cc:348] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.868255 6736 raft_consensus.cc:374] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.868301 6736 raft_consensus.cc:729] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.868407 6736 consensus_queue.cc:260] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.868618 6741 tablet_bootstrap.cc:654] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.868659 6736 ts_tablet_manager.cc:1398] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:34.869571 6741 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:34.869619 6737 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:34.869644 6741 ts_tablet_manager.cc:1367] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:34.869683 6737 ts_tablet_manager.cc:1367] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240501 19:57:34.870101 6738 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:34.870026 6741 raft_consensus.cc:348] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.870083 6737 raft_consensus.cc:348] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.870162 6741 raft_consensus.cc:374] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.870183 6737 raft_consensus.cc:374] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.870196 6741 raft_consensus.cc:729] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.870218 6737 raft_consensus.cc:729] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.870211 6738 ts_tablet_manager.cc:1367] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240501 19:57:34.870352 6737 consensus_queue.cc:260] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.870627 6737 ts_tablet_manager.cc:1398] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:34.870642 6738 raft_consensus.cc:348] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.870779 6738 raft_consensus.cc:374] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.870822 6738 raft_consensus.cc:729] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.870909 6741 consensus_queue.cc:260] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.871171 6741 ts_tablet_manager.cc:1398] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:34.871362 6738 consensus_queue.cc:260] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.871728 6738 ts_tablet_manager.cc:1398] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:34.872489 6741 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:34.873301 6741 tablet_bootstrap.cc:654] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:34.874167 6741 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:34.874271 6741 ts_tablet_manager.cc:1367] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 19:57:34.874537 6741 raft_consensus.cc:348] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.874611 6741 raft_consensus.cc:374] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:34.874629 6741 raft_consensus.cc:729] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:34.874729 6741 consensus_queue.cc:260] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.874918 6741 ts_tablet_manager.cc:1398] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:34.915553 6662 raft_consensus.cc:482] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:34.915632 6662 raft_consensus.cc:504] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.916152 6662 leader_election.cc:288] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:34.916306 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d29eb6552945a7879ee1a7e12919f0" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:34.916290 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d29eb6552945a7879ee1a7e12919f0" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:34.916407 5695 raft_consensus.cc:2454] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:34.916440 5828 raft_consensus.cc:2454] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:34.916510 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d29eb6552945a7879ee1a7e12919f0" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:34.916584 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d29eb6552945a7879ee1a7e12919f0" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:34.916620 5430 raft_consensus.cc:2454] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:34.916599 6095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d29eb6552945a7879ee1a7e12919f0" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:34.916690 5962 raft_consensus.cc:2454] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:34.916697 6095 raft_consensus.cc:2454] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:34.916934 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d29eb6552945a7879ee1a7e12919f0" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:34.916983 5233 leader_election.cc:302] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 9782508a47ba4403a39ca49fff7677fa, a881063bd86945a8871c7c102d8235c8, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:34.917057 5562 raft_consensus.cc:2454] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:34.917136 6662 raft_consensus.cc:2781] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:34.917183 6662 raft_consensus.cc:482] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:34.917217 6662 raft_consensus.cc:3037] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.917917 6662 raft_consensus.cc:504] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.918383 6662 leader_election.cc:288] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [CANDIDATE]: Term 1 election: Requested vote from peers 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:34.918550 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d29eb6552945a7879ee1a7e12919f0" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:34.918658 5828 raft_consensus.cc:3037] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.918675 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d29eb6552945a7879ee1a7e12919f0" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:34.918691 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d29eb6552945a7879ee1a7e12919f0" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:34.918751 5430 raft_consensus.cc:3037] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.918777 5962 raft_consensus.cc:3037] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.918761 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d29eb6552945a7879ee1a7e12919f0" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:34.918844 5695 raft_consensus.cc:3037] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.919584 5430 raft_consensus.cc:2454] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:34.919584 5828 raft_consensus.cc:2454] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:34.919665 5695 raft_consensus.cc:2454] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:34.919665 5962 raft_consensus.cc:2454] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:34.919799 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d29eb6552945a7879ee1a7e12919f0" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:34.919909 5562 raft_consensus.cc:3037] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.920009 6095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d29eb6552945a7879ee1a7e12919f0" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:34.920107 6095 raft_consensus.cc:3037] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.920295 5233 leader_election.cc:302] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:34.920447 6662 raft_consensus.cc:2781] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:34.920526 6662 raft_consensus.cc:686] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Becoming Leader. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Running, Role: LEADER
I20240501 19:57:34.920657 6662 consensus_queue.cc:237] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.920886 5562 raft_consensus.cc:2454] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:34.921083 6095 raft_consensus.cc:2454] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:34.922890 5147 catalog_manager.cc:5484] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:34.933071 6595 raft_consensus.cc:482] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:34.933145 6595 raft_consensus.cc:504] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.934207 6595 leader_election.cc:288] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:34.938613 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e17a6ae467bf42abac714a94e1ec28e5" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:34.938642 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e17a6ae467bf42abac714a94e1ec28e5" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:34.938721 5296 raft_consensus.cc:2454] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:34.938724 5695 raft_consensus.cc:2454] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:34.939396 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e17a6ae467bf42abac714a94e1ec28e5" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:34.939502 5430 raft_consensus.cc:2454] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:34.939777 5761 leader_election.cc:302] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:34.940040 6595 raft_consensus.cc:2781] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:34.940106 6595 raft_consensus.cc:482] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:34.940155 6595 raft_consensus.cc:3037] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.941330 6595 raft_consensus.cc:504] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.941865 6595 leader_election.cc:288] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [CANDIDATE]: Term 1 election: Requested vote from peers 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:34.941988 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e17a6ae467bf42abac714a94e1ec28e5" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:34.942095 5695 raft_consensus.cc:3037] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.942314 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e17a6ae467bf42abac714a94e1ec28e5" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:34.942309 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e17a6ae467bf42abac714a94e1ec28e5" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:34.942399 5430 raft_consensus.cc:3037] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.942399 5296 raft_consensus.cc:3037] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.942873 5695 raft_consensus.cc:2454] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:34.943166 5296 raft_consensus.cc:2454] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:34.943163 5430 raft_consensus.cc:2454] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:34.943467 5761 leader_election.cc:302] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:34.943651 6595 raft_consensus.cc:2781] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:34.943854 6595 raft_consensus.cc:686] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Becoming Leader. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Running, Role: LEADER
I20240501 19:57:34.944155 6595 consensus_queue.cc:237] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:34.944526 6095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e17a6ae467bf42abac714a94e1ec28e5" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:34.944620 6095 raft_consensus.cc:2454] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:34.944764 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e17a6ae467bf42abac714a94e1ec28e5" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:34.944846 6096 raft_consensus.cc:3037] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.945618 6096 raft_consensus.cc:2454] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:34.945626 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e17a6ae467bf42abac714a94e1ec28e5" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:34.945729 5562 raft_consensus.cc:2454] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:34.945904 5563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e17a6ae467bf42abac714a94e1ec28e5" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:34.945998 5563 raft_consensus.cc:3037] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.946729 5563 raft_consensus.cc:2454] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:34.947203 5147 catalog_manager.cc:5484] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5). New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:34.948920 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e17a6ae467bf42abac714a94e1ec28e5" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:34.949025 5962 raft_consensus.cc:2454] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:34.949177 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e17a6ae467bf42abac714a94e1ec28e5" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:34.949252 5960 raft_consensus.cc:3037] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.950060 5960 raft_consensus.cc:2454] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:34.969375 6603 raft_consensus.cc:482] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:34.969462 6603 raft_consensus.cc:504] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.970754 6603 leader_election.cc:288] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:34.975653 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681575c2bb142e0915be3dc75858f87" candidate_uuid: "9782508a47ba4403a39ca49fff7677fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:34.975720 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681575c2bb142e0915be3dc75858f87" candidate_uuid: "9782508a47ba4403a39ca49fff7677fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:34.975770 5695 raft_consensus.cc:2454] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9782508a47ba4403a39ca49fff7677fa in term 0.
I20240501 19:57:34.975826 5828 raft_consensus.cc:2454] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9782508a47ba4403a39ca49fff7677fa in term 0.
I20240501 19:57:34.976012 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681575c2bb142e0915be3dc75858f87" candidate_uuid: "9782508a47ba4403a39ca49fff7677fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:34.976126 5296 raft_consensus.cc:2454] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9782508a47ba4403a39ca49fff7677fa in term 0.
I20240501 19:57:34.976640 5363 leader_election.cc:302] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:34.976697 5563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681575c2bb142e0915be3dc75858f87" candidate_uuid: "9782508a47ba4403a39ca49fff7677fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:34.976799 5563 raft_consensus.cc:2454] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9782508a47ba4403a39ca49fff7677fa in term 0.
I20240501 19:57:34.976862 6603 raft_consensus.cc:2781] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:34.976902 6603 raft_consensus.cc:482] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:34.976929 6603 raft_consensus.cc:3037] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.977766 6603 raft_consensus.cc:504] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.978274 6603 leader_election.cc:288] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [CANDIDATE]: Term 1 election: Requested vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:34.978677 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681575c2bb142e0915be3dc75858f87" candidate_uuid: "9782508a47ba4403a39ca49fff7677fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:34.978683 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681575c2bb142e0915be3dc75858f87" candidate_uuid: "9782508a47ba4403a39ca49fff7677fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:34.978778 5828 raft_consensus.cc:3037] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.978785 5695 raft_consensus.cc:3037] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.978847 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681575c2bb142e0915be3dc75858f87" candidate_uuid: "9782508a47ba4403a39ca49fff7677fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:34.978889 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681575c2bb142e0915be3dc75858f87" candidate_uuid: "9782508a47ba4403a39ca49fff7677fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:34.978943 5960 raft_consensus.cc:2454] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9782508a47ba4403a39ca49fff7677fa in term 0.
I20240501 19:57:34.979269 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681575c2bb142e0915be3dc75858f87" candidate_uuid: "9782508a47ba4403a39ca49fff7677fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:34.979372 6096 raft_consensus.cc:2454] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9782508a47ba4403a39ca49fff7677fa in term 0.
I20240501 19:57:34.979550 6095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681575c2bb142e0915be3dc75858f87" candidate_uuid: "9782508a47ba4403a39ca49fff7677fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:34.979655 6095 raft_consensus.cc:3037] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.979641 5828 raft_consensus.cc:2454] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9782508a47ba4403a39ca49fff7677fa in term 1.
I20240501 19:57:34.979701 5695 raft_consensus.cc:2454] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9782508a47ba4403a39ca49fff7677fa in term 1.
I20240501 19:57:34.980013 5563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681575c2bb142e0915be3dc75858f87" candidate_uuid: "9782508a47ba4403a39ca49fff7677fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:34.980111 5563 raft_consensus.cc:3037] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.980499 6095 raft_consensus.cc:2454] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9782508a47ba4403a39ca49fff7677fa in term 1.
I20240501 19:57:34.980796 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0681575c2bb142e0915be3dc75858f87" candidate_uuid: "9782508a47ba4403a39ca49fff7677fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:34.980796 5366 leader_election.cc:302] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9782508a47ba4403a39ca49fff7677fa, a881063bd86945a8871c7c102d8235c8; no voters: 9544b0e566c54a62a6135f3bff47c4f2
I20240501 19:57:34.980834 5563 raft_consensus.cc:2454] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9782508a47ba4403a39ca49fff7677fa in term 1.
I20240501 19:57:34.980907 5296 raft_consensus.cc:3037] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:34.980964 6603 raft_consensus.cc:2781] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:34.981178 6603 raft_consensus.cc:686] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEADER]: Becoming Leader. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Running, Role: LEADER
I20240501 19:57:34.981462 6603 consensus_queue.cc:237] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:34.981774 5296 raft_consensus.cc:2454] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9782508a47ba4403a39ca49fff7677fa in term 1.
I20240501 19:57:34.983800 5147 catalog_manager.cc:5484] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 9782508a47ba4403a39ca49fff7677fa (127.20.210.2). New cstate: current_term: 1 leader_uuid: "9782508a47ba4403a39ca49fff7677fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } }
W20240501 19:57:35.117462 6784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:35.117656 6784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:35.120853 6784 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240501 19:57:35.197386 5563 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.197562 5430 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.197899 5962 raft_consensus.cc:3037] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.197970 5828 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.198289 5296 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.199036 5962 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.199630 5696 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.199648 6096 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.199688 5695 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.199833 6603 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.200042 6095 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.200284 6595 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.200510 5960 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.200542 5562 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.200896 5563 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.201093 6595 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.201395 6595 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.201498 5828 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.201941 6662 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.207288 6664 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.207331 6782 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.207553 6664 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.207592 6782 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.207815 6782 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.208000 6782 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.211453 5962 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.211776 6595 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.215795 5430 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.216056 5695 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.216529 5296 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.217226 6095 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.217586 6595 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.217825 6595 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.218056 6603 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.221325 6664 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.229034 6663 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.229349 6663 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 160.185 ms
time per row: 0.0800925 ms
I20240501 19:57:35.335904 5152 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47970:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 19:57:35.336445 5152 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_2 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 19:57:35.341569 5807 tablet_service.cc:1467] Processing CreateTablet for tablet 913563ed5f904cc28ff15148d9a5e5a2 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.341569 5673 tablet_service.cc:1467] Processing CreateTablet for tablet 913563ed5f904cc28ff15148d9a5e5a2 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.341827 5673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 913563ed5f904cc28ff15148d9a5e5a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.341822 5807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 913563ed5f904cc28ff15148d9a5e5a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.341898 5275 tablet_service.cc:1467] Processing CreateTablet for tablet 913563ed5f904cc28ff15148d9a5e5a2 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.342147 5275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 913563ed5f904cc28ff15148d9a5e5a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.342872 5408 tablet_service.cc:1467] Processing CreateTablet for tablet 913563ed5f904cc28ff15148d9a5e5a2 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.343027 5276 tablet_service.cc:1467] Processing CreateTablet for tablet d3937a2d841440d793ba2637f1af060f (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.343092 5408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 913563ed5f904cc28ff15148d9a5e5a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.343209 5276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3937a2d841440d793ba2637f1af060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.343809 6739 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:35.344563 6739 tablet_bootstrap.cc:654] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.345108 6742 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:35.345324 5409 tablet_service.cc:1467] Processing CreateTablet for tablet d3937a2d841440d793ba2637f1af060f (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.345523 5409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3937a2d841440d793ba2637f1af060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.345652 5807 tablet_service.cc:1467] Processing CreateTablet for tablet d3937a2d841440d793ba2637f1af060f (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.345829 6742 tablet_bootstrap.cc:654] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.345850 5807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3937a2d841440d793ba2637f1af060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.347057 6742 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:35.347154 6742 ts_tablet_manager.cc:1367] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:35.347213 6741 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:35.347756 6742 raft_consensus.cc:348] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.347946 6741 tablet_bootstrap.cc:654] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.347970 6742 raft_consensus.cc:374] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.348007 6742 raft_consensus.cc:729] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.348075 6739 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:35.348140 6739 ts_tablet_manager.cc:1367] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240501 19:57:35.348125 6742 consensus_queue.cc:260] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.348381 6742 ts_tablet_manager.cc:1398] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:35.348420 6738 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:35.348512 6742 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:35.348513 6739 raft_consensus.cc:348] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.348608 6739 raft_consensus.cc:374] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.348637 6739 raft_consensus.cc:729] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.348734 6739 consensus_queue.cc:260] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.348982 6739 ts_tablet_manager.cc:1398] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:35.349113 6739 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:35.349121 6738 tablet_bootstrap.cc:654] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.349253 6742 tablet_bootstrap.cc:654] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.349938 6739 tablet_bootstrap.cc:654] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.350849 6075 tablet_service.cc:1467] Processing CreateTablet for tablet 913563ed5f904cc28ff15148d9a5e5a2 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.350929 5673 tablet_service.cc:1467] Processing CreateTablet for tablet d3937a2d841440d793ba2637f1af060f (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.351083 6075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 913563ed5f904cc28ff15148d9a5e5a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.351038 5942 tablet_service.cc:1467] Processing CreateTablet for tablet 913563ed5f904cc28ff15148d9a5e5a2 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.351135 5673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3937a2d841440d793ba2637f1af060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.351282 5942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 913563ed5f904cc28ff15148d9a5e5a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.353073 6737 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:35.353214 6739 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:35.353284 6739 ts_tablet_manager.cc:1367] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:57:35.353233 6077 tablet_service.cc:1467] Processing CreateTablet for tablet d3937a2d841440d793ba2637f1af060f (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.353338 5941 tablet_service.cc:1467] Processing CreateTablet for tablet d3937a2d841440d793ba2637f1af060f (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.353432 6077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3937a2d841440d793ba2637f1af060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.353515 5941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3937a2d841440d793ba2637f1af060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.353647 6739 raft_consensus.cc:348] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.353773 6739 raft_consensus.cc:374] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.353809 6739 raft_consensus.cc:729] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.353817 6737 tablet_bootstrap.cc:654] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.353952 6739 consensus_queue.cc:260] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.354418 6739 ts_tablet_manager.cc:1398] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:35.354735 6737 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:35.354817 6737 ts_tablet_manager.cc:1367] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:57:35.355084 6737 raft_consensus.cc:348] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.355156 6737 raft_consensus.cc:374] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.355144 6735 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:35.355175 6737 raft_consensus.cc:729] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.355273 6737 consensus_queue.cc:260] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.355301 5543 tablet_service.cc:1467] Processing CreateTablet for tablet 913563ed5f904cc28ff15148d9a5e5a2 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.355502 6737 ts_tablet_manager.cc:1398] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:35.355530 5543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 913563ed5f904cc28ff15148d9a5e5a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.355654 6737 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:35.355947 6735 tablet_bootstrap.cc:654] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.356618 6737 tablet_bootstrap.cc:654] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.357498 6737 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:35.357578 6737 ts_tablet_manager.cc:1367] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:57:35.357875 6736 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:35.357986 6737 raft_consensus.cc:348] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.358094 6737 raft_consensus.cc:374] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.358129 6737 raft_consensus.cc:729] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.358223 6737 consensus_queue.cc:260] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.358495 6737 ts_tablet_manager.cc:1398] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:35.358664 6736 tablet_bootstrap.cc:654] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.359612 5542 tablet_service.cc:1467] Processing CreateTablet for tablet 4a713bc0643d4bdf91f669dde94f63f8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.359717 5275 tablet_service.cc:1467] Processing CreateTablet for tablet 4a713bc0643d4bdf91f669dde94f63f8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.359719 5409 tablet_service.cc:1467] Processing CreateTablet for tablet 4a713bc0643d4bdf91f669dde94f63f8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.359850 5542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a713bc0643d4bdf91f669dde94f63f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.359898 5275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a713bc0643d4bdf91f669dde94f63f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.359968 5409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a713bc0643d4bdf91f669dde94f63f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.361894 5543 tablet_service.cc:1467] Processing CreateTablet for tablet d3937a2d841440d793ba2637f1af060f (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.362114 5543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3937a2d841440d793ba2637f1af060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.362823 6606 raft_consensus.cc:482] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:35.362907 6606 raft_consensus.cc:504] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.363178 5673 tablet_service.cc:1467] Processing CreateTablet for tablet 4a713bc0643d4bdf91f669dde94f63f8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.363219 5807 tablet_service.cc:1467] Processing CreateTablet for tablet 4a713bc0643d4bdf91f669dde94f63f8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.363373 5673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a713bc0643d4bdf91f669dde94f63f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.363440 5807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a713bc0643d4bdf91f669dde94f63f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.364306 6741 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:35.364377 6741 ts_tablet_manager.cc:1367] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.001s
I20240501 19:57:35.364442 6606 leader_election.cc:288] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:35.364739 6741 raft_consensus.cc:348] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.364876 6741 raft_consensus.cc:374] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.364907 6741 raft_consensus.cc:729] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.365039 6741 consensus_queue.cc:260] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.365089 6739 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:35.365299 6741 ts_tablet_manager.cc:1398] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:35.365438 6741 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:35.365872 6739 tablet_bootstrap.cc:654] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.366014 5941 tablet_service.cc:1467] Processing CreateTablet for tablet 4a713bc0643d4bdf91f669dde94f63f8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.366096 6741 tablet_bootstrap.cc:654] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.366215 5941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a713bc0643d4bdf91f669dde94f63f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.369473 6077 tablet_service.cc:1467] Processing CreateTablet for tablet 4a713bc0643d4bdf91f669dde94f63f8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:35.369722 6077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a713bc0643d4bdf91f669dde94f63f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:35.371207 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "a881063bd86945a8871c7c102d8235c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:35.371322 5296 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a881063bd86945a8871c7c102d8235c8 in term 0.
I20240501 19:57:35.371902 6737 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:35.372653 6737 tablet_bootstrap.cc:654] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.372748 6738 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:35.372823 6738 ts_tablet_manager.cc:1367] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.002s
I20240501 19:57:35.373207 6738 raft_consensus.cc:348] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.373343 6738 raft_consensus.cc:374] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.373378 6738 raft_consensus.cc:729] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.373479 6738 consensus_queue.cc:260] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.373714 6738 ts_tablet_manager.cc:1398] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:35.373854 6738 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:35.374655 6738 tablet_bootstrap.cc:654] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.375331 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "a881063bd86945a8871c7c102d8235c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:35.375423 5827 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a881063bd86945a8871c7c102d8235c8 in term 0.
I20240501 19:57:35.376114 5428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "a881063bd86945a8871c7c102d8235c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:35.376479 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "a881063bd86945a8871c7c102d8235c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
W20240501 19:57:35.376526 6030 leader_election.cc:341] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:57:35.376804 6737 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
W20240501 19:57:35.376868 6033 leader_election.cc:341] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:57:35.376997 6735 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:35.376878 6737 ts_tablet_manager.cc:1367] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 19:57:35.377084 6735 ts_tablet_manager.cc:1367] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20240501 19:57:35.377267 6736 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:35.377337 6736 ts_tablet_manager.cc:1367] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20240501 19:57:35.377456 6737 raft_consensus.cc:348] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.377545 6737 raft_consensus.cc:374] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.377574 6737 raft_consensus.cc:729] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.377530 6735 raft_consensus.cc:348] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.377637 6735 raft_consensus.cc:374] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.377668 6735 raft_consensus.cc:729] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.377671 6736 raft_consensus.cc:348] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.377800 6736 raft_consensus.cc:374] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.377830 6736 raft_consensus.cc:729] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.377794 6735 consensus_queue.cc:260] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.377940 6736 consensus_queue.cc:260] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.377971 6737 consensus_queue.cc:260] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.378060 6735 ts_tablet_manager.cc:1398] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:35.378136 6741 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:35.378201 6735 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:35.378202 6741 ts_tablet_manager.cc:1367] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240501 19:57:35.378194 6736 ts_tablet_manager.cc:1398] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:35.378381 6736 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:35.378638 6741 raft_consensus.cc:348] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.378751 6741 raft_consensus.cc:374] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.378778 6741 raft_consensus.cc:729] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.378880 6741 consensus_queue.cc:260] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.379035 6735 tablet_bootstrap.cc:654] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.379164 6741 ts_tablet_manager.cc:1398] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:35.379182 6736 tablet_bootstrap.cc:654] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.379323 6741 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:35.379592 6737 ts_tablet_manager.cc:1398] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:35.379906 6742 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:35.380004 6742 ts_tablet_manager.cc:1367] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.000s
I20240501 19:57:35.380088 6741 tablet_bootstrap.cc:654] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.380373 6742 raft_consensus.cc:348] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.380468 6742 raft_consensus.cc:374] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.380503 6742 raft_consensus.cc:729] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.380632 6742 consensus_queue.cc:260] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.380903 6742 ts_tablet_manager.cc:1398] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:35.381059 6742 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:35.381982 6738 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:35.382045 6738 ts_tablet_manager.cc:1367] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240501 19:57:35.382083 6742 tablet_bootstrap.cc:654] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.382462 6738 raft_consensus.cc:348] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.382555 6738 raft_consensus.cc:374] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.382583 6738 raft_consensus.cc:729] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.382704 6738 consensus_queue.cc:260] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.382851 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "a881063bd86945a8871c7c102d8235c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:35.382944 6738 ts_tablet_manager.cc:1398] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:35.383076 6738 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:35.383601 6739 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:35.383675 6739 ts_tablet_manager.cc:1367] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20240501 19:57:35.383900 6738 tablet_bootstrap.cc:654] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.384095 6739 raft_consensus.cc:348] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.384229 6739 raft_consensus.cc:374] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.384266 6739 raft_consensus.cc:729] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.384370 6739 consensus_queue.cc:260] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.384627 6739 ts_tablet_manager.cc:1398] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240501 19:57:35.384939 6033 leader_election.cc:341] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:57:35.385080 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "a881063bd86945a8871c7c102d8235c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:35.385360 6735 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:35.385425 6735 ts_tablet_manager.cc:1367] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
W20240501 19:57:35.385552 6033 leader_election.cc:341] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:57:35.385610 6033 leader_election.cc:302] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, a881063bd86945a8871c7c102d8235c8, ad52038f1d38489ea464a8fbc72daac0; no voters: 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, 9782508a47ba4403a39ca49fff7677fa, f5287d5ae4d84bd692b0f7985cd94171
I20240501 19:57:35.385818 6735 raft_consensus.cc:348] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.385912 6735 raft_consensus.cc:374] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.385944 6735 raft_consensus.cc:729] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.385970 6606 raft_consensus.cc:2726] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240501 19:57:35.386045 6735 consensus_queue.cc:260] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.386346 6735 ts_tablet_manager.cc:1398] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:35.386482 6735 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:35.386571 6738 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:35.386631 6738 ts_tablet_manager.cc:1367] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:57:35.387029 6738 raft_consensus.cc:348] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.387157 6738 raft_consensus.cc:374] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.387197 6738 raft_consensus.cc:729] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.387266 6735 tablet_bootstrap.cc:654] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.387329 6738 consensus_queue.cc:260] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.387626 6738 ts_tablet_manager.cc:1398] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:35.387872 6736 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:35.387943 6736 ts_tablet_manager.cc:1367] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240501 19:57:35.388305 6736 raft_consensus.cc:348] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.388422 6736 raft_consensus.cc:374] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.388458 6736 raft_consensus.cc:729] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.388561 6736 consensus_queue.cc:260] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.388849 6736 ts_tablet_manager.cc:1398] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:35.388945 6735 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:35.388988 6736 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:35.389010 6735 ts_tablet_manager.cc:1367] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:57:35.389047 6742 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:35.389114 6742 ts_tablet_manager.cc:1367] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240501 19:57:35.389364 6735 raft_consensus.cc:348] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.389461 6735 raft_consensus.cc:374] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.389496 6735 raft_consensus.cc:729] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.389470 6742 raft_consensus.cc:348] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.389564 6742 raft_consensus.cc:374] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.389600 6742 raft_consensus.cc:729] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.389623 6735 consensus_queue.cc:260] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.389695 6742 consensus_queue.cc:260] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.389724 6736 tablet_bootstrap.cc:654] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:35.389945 6742 ts_tablet_manager.cc:1398] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:35.389955 6735 ts_tablet_manager.cc:1398] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:35.390780 6736 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:35.390856 6736 ts_tablet_manager.cc:1367] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:35.391271 6736 raft_consensus.cc:348] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.391387 6736 raft_consensus.cc:374] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.391417 6736 raft_consensus.cc:729] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.391506 6736 consensus_queue.cc:260] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.391747 6736 ts_tablet_manager.cc:1398] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:35.391793 6741 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:35.391862 6741 ts_tablet_manager.cc:1367] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240501 19:57:35.392194 6741 raft_consensus.cc:348] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.392298 6741 raft_consensus.cc:374] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:35.392325 6741 raft_consensus.cc:729] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:35.392413 6741 consensus_queue.cc:260] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.392684 6741 ts_tablet_manager.cc:1398] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:35.496193 6662 raft_consensus.cc:482] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:35.496290 6662 raft_consensus.cc:504] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.496807 6662 leader_election.cc:288] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:35.497040 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "913563ed5f904cc28ff15148d9a5e5a2" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:35.497038 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "913563ed5f904cc28ff15148d9a5e5a2" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:35.497161 5696 raft_consensus.cc:2454] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:35.497167 5827 raft_consensus.cc:2454] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:35.497231 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "913563ed5f904cc28ff15148d9a5e5a2" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:35.497339 5562 raft_consensus.cc:2454] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:35.497392 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "913563ed5f904cc28ff15148d9a5e5a2" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:35.497392 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "913563ed5f904cc28ff15148d9a5e5a2" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:35.497503 5960 raft_consensus.cc:2454] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:35.497503 6096 raft_consensus.cc:2454] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:35.497764 5428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "913563ed5f904cc28ff15148d9a5e5a2" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:35.497786 5233 leader_election.cc:302] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 4d4855c096ac4e1e936abf15834332ae, a881063bd86945a8871c7c102d8235c8, ad52038f1d38489ea464a8fbc72daac0, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:35.497890 5428 raft_consensus.cc:2454] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 0.
I20240501 19:57:35.497938 6662 raft_consensus.cc:2781] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:35.497978 6662 raft_consensus.cc:482] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:35.498030 6662 raft_consensus.cc:3037] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.498816 6662 raft_consensus.cc:504] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.499254 6662 leader_election.cc:288] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [CANDIDATE]: Term 1 election: Requested vote from peers 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:35.499500 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "913563ed5f904cc28ff15148d9a5e5a2" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:35.499517 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "913563ed5f904cc28ff15148d9a5e5a2" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:35.499565 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "913563ed5f904cc28ff15148d9a5e5a2" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:35.499563 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "913563ed5f904cc28ff15148d9a5e5a2" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:35.499593 5960 raft_consensus.cc:3037] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.499593 5827 raft_consensus.cc:3037] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.499647 5696 raft_consensus.cc:3037] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.499655 6096 raft_consensus.cc:3037] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.500504 5827 raft_consensus.cc:2454] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:35.500504 5960 raft_consensus.cc:2454] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:35.500583 5696 raft_consensus.cc:2454] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:35.500583 6096 raft_consensus.cc:2454] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:35.500874 5233 leader_election.cc:302] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 9544b0e566c54a62a6135f3bff47c4f2, a881063bd86945a8871c7c102d8235c8, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:35.500938 5428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "913563ed5f904cc28ff15148d9a5e5a2" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:35.501041 5428 raft_consensus.cc:3037] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.501083 6662 raft_consensus.cc:2781] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:35.501142 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "913563ed5f904cc28ff15148d9a5e5a2" candidate_uuid: "ad52038f1d38489ea464a8fbc72daac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:35.501178 6662 raft_consensus.cc:686] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Becoming Leader. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Running, Role: LEADER
I20240501 19:57:35.501262 5562 raft_consensus.cc:3037] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.501335 6662 consensus_queue.cc:237] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.501945 5428 raft_consensus.cc:2454] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:35.502123 5562 raft_consensus.cc:2454] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad52038f1d38489ea464a8fbc72daac0 in term 1.
I20240501 19:57:35.503324 5154 catalog_manager.cc:5484] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1). New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:35.516288 6815 raft_consensus.cc:482] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:35.516371 6815 raft_consensus.cc:504] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.516849 6815 leader_election.cc:288] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:35.517130 5428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a713bc0643d4bdf91f669dde94f63f8" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:35.517179 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a713bc0643d4bdf91f669dde94f63f8" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:35.517228 5428 raft_consensus.cc:2454] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:35.517216 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a713bc0643d4bdf91f669dde94f63f8" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:35.517226 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a713bc0643d4bdf91f669dde94f63f8" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:35.517266 5562 raft_consensus.cc:2454] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:35.517331 5296 raft_consensus.cc:2454] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:35.517331 6096 raft_consensus.cc:2454] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:35.517580 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a713bc0643d4bdf91f669dde94f63f8" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:35.517601 5764 leader_election.cc:302] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 9782508a47ba4403a39ca49fff7677fa, a881063bd86945a8871c7c102d8235c8, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:35.517685 5960 raft_consensus.cc:2454] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:35.517802 6815 raft_consensus.cc:2781] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:35.517807 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a713bc0643d4bdf91f669dde94f63f8" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:35.517850 6815 raft_consensus.cc:482] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:35.517885 6815 raft_consensus.cc:3037] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.517906 5696 raft_consensus.cc:2454] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:35.518625 6815 raft_consensus.cc:504] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.519083 6815 leader_election.cc:288] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [CANDIDATE]: Term 1 election: Requested vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:35.519197 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a713bc0643d4bdf91f669dde94f63f8" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:35.519284 5296 raft_consensus.cc:3037] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.519300 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a713bc0643d4bdf91f669dde94f63f8" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:35.519389 5562 raft_consensus.cc:3037] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.519522 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a713bc0643d4bdf91f669dde94f63f8" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:35.519625 6096 raft_consensus.cc:3037] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.519773 5428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a713bc0643d4bdf91f669dde94f63f8" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:35.519862 5428 raft_consensus.cc:3037] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.520272 5296 raft_consensus.cc:2454] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:35.520291 5562 raft_consensus.cc:2454] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:35.520546 6096 raft_consensus.cc:2454] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:35.520792 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a713bc0643d4bdf91f669dde94f63f8" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:35.520817 5428 raft_consensus.cc:2454] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:35.520834 5764 leader_election.cc:302] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, a881063bd86945a8871c7c102d8235c8, ad52038f1d38489ea464a8fbc72daac0, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:35.520891 5960 raft_consensus.cc:3037] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.521008 6815 raft_consensus.cc:2781] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:35.521090 6815 raft_consensus.cc:686] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Becoming Leader. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Running, Role: LEADER
I20240501 19:57:35.521209 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a713bc0643d4bdf91f669dde94f63f8" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:35.521225 6815 consensus_queue.cc:237] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:35.521306 5696 raft_consensus.cc:3037] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.521793 5960 raft_consensus.cc:2454] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:35.522069 5696 raft_consensus.cc:2454] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:35.523140 5154 catalog_manager.cc:5484] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5). New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:35.658744 6724 raft_consensus.cc:482] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:35.658856 6724 raft_consensus.cc:504] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.659391 6724 leader_election.cc:288] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
I20240501 19:57:35.659691 5428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:35.659751 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:35.659816 5428 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:35.659858 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:35.659899 5827 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:35.659996 5696 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:35.660275 5499 leader_election.cc:302] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9782508a47ba4403a39ca49fff7677fa, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:35.660331 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:35.660459 6096 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:35.660486 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:35.660532 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:35.660590 5960 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:35.660632 6724 raft_consensus.cc:2781] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:35.660653 5296 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:35.660689 6724 raft_consensus.cc:482] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:35.660724 6724 raft_consensus.cc:3037] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.661641 6724 raft_consensus.cc:504] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.662106 6724 leader_election.cc:288] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Requested vote from peers 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
I20240501 19:57:35.662322 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:35.662382 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:35.662411 5827 raft_consensus.cc:3037] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.662474 5696 raft_consensus.cc:3037] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.662499 5428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:35.662533 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:35.662583 5428 raft_consensus.cc:3037] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.662631 5960 raft_consensus.cc:3037] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.663270 5827 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:35.663313 5696 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:35.663390 5428 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:35.663477 5960 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:35.663678 5499 leader_election.cc:302] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9782508a47ba4403a39ca49fff7677fa, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:35.663658 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:35.663810 5296 raft_consensus.cc:3037] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.663838 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3937a2d841440d793ba2637f1af060f" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:35.663895 6724 raft_consensus.cc:2781] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:35.663950 6096 raft_consensus.cc:3037] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:35.663978 6724 raft_consensus.cc:686] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Becoming Leader. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Running, Role: LEADER
I20240501 19:57:35.664126 6724 consensus_queue.cc:237] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:35.664717 5296 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:35.664850 6096 raft_consensus.cc:2454] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:35.666102 5154 catalog_manager.cc:5484] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } }
W20240501 19:57:35.823136 6872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:35.823320 6872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:35.826484 6872 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240501 19:57:35.891917 6096 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.891917 5428 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.892252 6724 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.892580 6725 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.893121 6664 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.893846 5428 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.894170 6095 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.894271 5696 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.894407 5561 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.894999 5696 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.895313 6664 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.895505 6725 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.895548 6664 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.895787 6664 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.897290 5962 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.897670 6725 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.897727 6662 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.898100 5828 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.899607 6724 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.901392 6662 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.901507 5296 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.902810 6724 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.952193 6815 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.959149 5696 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.959517 6816 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.961640 5430 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.961953 6816 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.968119 5960 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.968472 6816 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.970885 5561 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.971298 6816 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:35.975489 5296 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:35.975800 6816 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 179.016 ms
time per row: 0.0895079 ms
I20240501 19:57:36.040536 5154 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47970:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 19:57:36.040995 5154 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_3 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 19:57:36.045629 5275 tablet_service.cc:1467] Processing CreateTablet for tablet d8a84349ba37496289351ac689f77eab (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.045643 5409 tablet_service.cc:1467] Processing CreateTablet for tablet d8a84349ba37496289351ac689f77eab (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.045899 5409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8a84349ba37496289351ac689f77eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.045900 5275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8a84349ba37496289351ac689f77eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.046829 5807 tablet_service.cc:1467] Processing CreateTablet for tablet d8a84349ba37496289351ac689f77eab (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.047075 5807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8a84349ba37496289351ac689f77eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.047617 5809 tablet_service.cc:1467] Processing CreateTablet for tablet f27a79cf3a044383918f5d6f600a0fd9 (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.047830 5809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f27a79cf3a044383918f5d6f600a0fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.048133 5543 tablet_service.cc:1467] Processing CreateTablet for tablet d8a84349ba37496289351ac689f77eab (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.048359 5543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8a84349ba37496289351ac689f77eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.048830 6977 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:36.048986 5808 tablet_service.cc:1467] Processing CreateTablet for tablet 064f698b376e407da78c7d439553896a (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.049170 5808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064f698b376e407da78c7d439553896a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.049620 6977 tablet_bootstrap.cc:654] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.049782 6976 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:36.050472 6979 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:36.050635 6976 tablet_bootstrap.cc:654] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.050699 6975 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:36.051332 6979 tablet_bootstrap.cc:654] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.051566 6975 tablet_bootstrap.cc:654] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.051754 6976 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:36.051824 6976 ts_tablet_manager.cc:1367] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:36.052181 5409 tablet_service.cc:1467] Processing CreateTablet for tablet 064f698b376e407da78c7d439553896a (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.052260 6976 raft_consensus.cc:348] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.052376 6976 raft_consensus.cc:374] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.052405 6976 raft_consensus.cc:729] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.052417 5409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064f698b376e407da78c7d439553896a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.052520 6976 consensus_queue.cc:260] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.052635 6975 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:36.052693 6975 ts_tablet_manager.cc:1367] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:36.052671 5275 tablet_service.cc:1467] Processing CreateTablet for tablet 064f698b376e407da78c7d439553896a (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.052804 6976 ts_tablet_manager.cc:1398] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.052861 5275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064f698b376e407da78c7d439553896a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.053047 6975 raft_consensus.cc:348] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.053120 6975 raft_consensus.cc:374] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.053138 6975 raft_consensus.cc:729] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.053225 6975 consensus_queue.cc:260] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.053437 6975 ts_tablet_manager.cc:1398] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:36.053921 6979 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:36.053987 6979 ts_tablet_manager.cc:1367] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:57:36.054009 6976 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:36.054399 6975 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:36.054373 6979 raft_consensus.cc:348] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.054483 6979 raft_consensus.cc:374] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.054510 6979 raft_consensus.cc:729] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.054553 5276 tablet_service.cc:1467] Processing CreateTablet for tablet f27a79cf3a044383918f5d6f600a0fd9 (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.054740 5276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f27a79cf3a044383918f5d6f600a0fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.054751 6976 tablet_bootstrap.cc:654] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.055085 6975 tablet_bootstrap.cc:654] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.054627 6979 consensus_queue.cc:260] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.055711 6979 ts_tablet_manager.cc:1398] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:36.056452 6975 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:36.056449 6976 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:36.056516 6975 ts_tablet_manager.cc:1367] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:36.056516 6976 ts_tablet_manager.cc:1367] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:57:36.052181 5408 tablet_service.cc:1467] Processing CreateTablet for tablet f27a79cf3a044383918f5d6f600a0fd9 (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.056977 6975 raft_consensus.cc:348] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.057061 6975 raft_consensus.cc:374] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.057087 6975 raft_consensus.cc:729] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.057114 5408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f27a79cf3a044383918f5d6f600a0fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.057204 6975 consensus_queue.cc:260] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.057521 6975 ts_tablet_manager.cc:1398] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.056895 6976 raft_consensus.cc:348] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.057642 6975 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:36.057576 6976 raft_consensus.cc:374] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.057930 6976 raft_consensus.cc:729] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.058058 6976 consensus_queue.cc:260] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.058347 6976 ts_tablet_manager.cc:1398] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:36.058415 6975 tablet_bootstrap.cc:654] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.058734 6976 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:36.059284 6976 tablet_bootstrap.cc:654] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.061751 5542 tablet_service.cc:1467] Processing CreateTablet for tablet f27a79cf3a044383918f5d6f600a0fd9 (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.061954 5941 tablet_service.cc:1467] Processing CreateTablet for tablet d8a84349ba37496289351ac689f77eab (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.062053 5542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f27a79cf3a044383918f5d6f600a0fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.062227 5941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8a84349ba37496289351ac689f77eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.062551 5942 tablet_service.cc:1467] Processing CreateTablet for tablet 064f698b376e407da78c7d439553896a (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.062738 5942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064f698b376e407da78c7d439553896a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.061751 5543 tablet_service.cc:1467] Processing CreateTablet for tablet 064f698b376e407da78c7d439553896a (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.063875 5543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064f698b376e407da78c7d439553896a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.064538 5940 tablet_service.cc:1467] Processing CreateTablet for tablet f27a79cf3a044383918f5d6f600a0fd9 (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.064728 5940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f27a79cf3a044383918f5d6f600a0fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.063611 6979 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:36.065837 6979 tablet_bootstrap.cc:654] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.066147 5673 tablet_service.cc:1467] Processing CreateTablet for tablet d8a84349ba37496289351ac689f77eab (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.066411 5673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8a84349ba37496289351ac689f77eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.066617 6979 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:36.066563 6076 tablet_service.cc:1467] Processing CreateTablet for tablet f27a79cf3a044383918f5d6f600a0fd9 (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.066666 6979 ts_tablet_manager.cc:1367] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:57:36.066805 6076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f27a79cf3a044383918f5d6f600a0fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.066982 6979 raft_consensus.cc:348] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.067072 6979 raft_consensus.cc:374] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.067090 6979 raft_consensus.cc:729] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.067191 6979 consensus_queue.cc:260] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.067406 6979 ts_tablet_manager.cc:1398] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:36.067744 6979 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:36.068634 6979 tablet_bootstrap.cc:654] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.068836 6989 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:36.068837 6075 tablet_service.cc:1467] Processing CreateTablet for tablet 064f698b376e407da78c7d439553896a (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.068885 6990 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:36.069039 6075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064f698b376e407da78c7d439553896a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.069576 6989 tablet_bootstrap.cc:654] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.070546 6989 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:36.070624 6989 ts_tablet_manager.cc:1367] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:36.070693 6990 tablet_bootstrap.cc:654] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.071050 6077 tablet_service.cc:1467] Processing CreateTablet for tablet d8a84349ba37496289351ac689f77eab (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.071055 6989 raft_consensus.cc:348] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.071172 6989 raft_consensus.cc:374] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.071203 6989 raft_consensus.cc:729] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.071300 6077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8a84349ba37496289351ac689f77eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.071341 6989 consensus_queue.cc:260] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.071584 6989 ts_tablet_manager.cc:1398] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.071683 6990 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:36.071748 6990 ts_tablet_manager.cc:1367] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240501 19:57:36.072091 6990 raft_consensus.cc:348] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.072189 6990 raft_consensus.cc:374] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.072219 6990 raft_consensus.cc:729] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.072800 5675 tablet_service.cc:1467] Processing CreateTablet for tablet f27a79cf3a044383918f5d6f600a0fd9 (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.072971 5675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f27a79cf3a044383918f5d6f600a0fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.073176 6990 consensus_queue.cc:260] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.073449 6990 ts_tablet_manager.cc:1398] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:57:36.073562 6990 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:36.074101 6990 tablet_bootstrap.cc:654] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.074534 6976 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:36.074599 6976 ts_tablet_manager.cc:1367] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240501 19:57:36.074815 5674 tablet_service.cc:1467] Processing CreateTablet for tablet 064f698b376e407da78c7d439553896a (DEFAULT_TABLE table=rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.074999 5674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064f698b376e407da78c7d439553896a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.074959 6976 raft_consensus.cc:348] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.075029 6990 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:36.075052 6976 raft_consensus.cc:374] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.075081 6976 raft_consensus.cc:729] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.075093 6990 ts_tablet_manager.cc:1367] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:36.075181 6976 consensus_queue.cc:260] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.075409 6976 ts_tablet_manager.cc:1398] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.075435 6990 raft_consensus.cc:348] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.075523 6990 raft_consensus.cc:374] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.075549 6990 raft_consensus.cc:729] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.075641 6990 consensus_queue.cc:260] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.075731 6989 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:36.075888 6990 ts_tablet_manager.cc:1398] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.076027 6990 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:36.076453 6989 tablet_bootstrap.cc:654] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.076769 6990 tablet_bootstrap.cc:654] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.077602 6987 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:36.078855 6987 tablet_bootstrap.cc:654] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.078984 6989 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:36.079069 6989 ts_tablet_manager.cc:1367] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:57:36.079476 6977 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:36.079466 6989 raft_consensus.cc:348] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.079560 6977 ts_tablet_manager.cc:1367] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20240501 19:57:36.079566 6989 raft_consensus.cc:374] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.079595 6989 raft_consensus.cc:729] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.079699 6989 consensus_queue.cc:260] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.079972 6989 ts_tablet_manager.cc:1398] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.080039 6977 raft_consensus.cc:348] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.080139 6989 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:36.080152 6977 raft_consensus.cc:374] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.080184 6977 raft_consensus.cc:729] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.080309 6977 consensus_queue.cc:260] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.080612 6977 ts_tablet_manager.cc:1398] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.080811 6977 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:36.081270 6989 tablet_bootstrap.cc:654] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.081661 6977 tablet_bootstrap.cc:654] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.082650 6979 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:36.082672 6977 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:36.082737 6977 ts_tablet_manager.cc:1367] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:36.082770 6979 ts_tablet_manager.cc:1367] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240501 19:57:36.082854 6975 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:36.082940 6975 ts_tablet_manager.cc:1367] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.000s
I20240501 19:57:36.083118 6977 raft_consensus.cc:348] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.083211 6977 raft_consensus.cc:374] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.083181 6979 raft_consensus.cc:348] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.083248 6977 raft_consensus.cc:729] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.083276 6979 raft_consensus.cc:374] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.083323 6979 raft_consensus.cc:729] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.083310 6975 raft_consensus.cc:348] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.083420 6975 raft_consensus.cc:374] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.083364 6977 consensus_queue.cc:260] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.083460 6975 raft_consensus.cc:729] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.083428 6979 consensus_queue.cc:260] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.083561 6975 consensus_queue.cc:260] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.083671 6977 ts_tablet_manager.cc:1398] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.083688 6979 ts_tablet_manager.cc:1398] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.083859 6975 ts_tablet_manager.cc:1398] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.083870 6977 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:36.085094 6977 tablet_bootstrap.cc:654] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.086771 6987 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:36.087728 6987 ts_tablet_manager.cc:1367] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240501 19:57:36.088073 6987 raft_consensus.cc:348] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.088163 6987 raft_consensus.cc:374] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.088182 6987 raft_consensus.cc:729] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.088285 6987 consensus_queue.cc:260] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.088557 6987 ts_tablet_manager.cc:1398] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:36.088747 6987 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:36.089393 6987 tablet_bootstrap.cc:654] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.090766 6987 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:36.090860 6987 ts_tablet_manager.cc:1367] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:36.091446 6987 raft_consensus.cc:348] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.091560 6987 raft_consensus.cc:374] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.091596 6987 raft_consensus.cc:729] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.091722 6987 consensus_queue.cc:260] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.091976 6987 ts_tablet_manager.cc:1398] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.092144 6987 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:36.092949 6987 tablet_bootstrap.cc:654] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.093899 6987 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:36.093973 6987 ts_tablet_manager.cc:1367] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:36.094305 6987 raft_consensus.cc:348] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.094410 6987 raft_consensus.cc:374] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.094441 6987 raft_consensus.cc:729] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.094542 6987 consensus_queue.cc:260] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.094774 6987 ts_tablet_manager.cc:1398] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.095057 6711 raft_consensus.cc:482] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:36.095136 6711 raft_consensus.cc:504] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.095620 6711 leader_election.cc:288] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:36.096449 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f27a79cf3a044383918f5d6f600a0fd9" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:36.096549 5827 raft_consensus.cc:2454] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.096525 5561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f27a79cf3a044383918f5d6f600a0fd9" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:36.096525 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f27a79cf3a044383918f5d6f600a0fd9" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:36.096623 6096 raft_consensus.cc:2454] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.096623 5561 raft_consensus.cc:2454] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.096907 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f27a79cf3a044383918f5d6f600a0fd9" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:36.097005 5296 raft_consensus.cc:2454] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.097194 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f27a79cf3a044383918f5d6f600a0fd9" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:36.097401 5430 raft_consensus.cc:2454] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.097653 5628 leader_election.cc:302] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:36.097721 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f27a79cf3a044383918f5d6f600a0fd9" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:36.097812 5960 raft_consensus.cc:2454] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.098012 6711 raft_consensus.cc:2781] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:36.098054 6711 raft_consensus.cc:482] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:36.098084 6711 raft_consensus.cc:3037] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.098481 6989 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:36.098481 6990 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:36.098562 6990 ts_tablet_manager.cc:1367] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.001s
I20240501 19:57:36.098560 6989 ts_tablet_manager.cc:1367] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240501 19:57:36.098940 6990 raft_consensus.cc:348] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.098940 6989 raft_consensus.cc:348] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.099052 6990 raft_consensus.cc:374] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.099052 6989 raft_consensus.cc:374] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.099084 6989 raft_consensus.cc:729] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.099084 6990 raft_consensus.cc:729] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.099205 6990 consensus_queue.cc:260] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.099195 6989 consensus_queue.cc:260] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.099462 6990 ts_tablet_manager.cc:1398] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:36.099539 6989 ts_tablet_manager.cc:1398] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.100462 6711 raft_consensus.cc:504] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.100965 6711 leader_election.cc:288] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 election: Requested vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:36.101130 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f27a79cf3a044383918f5d6f600a0fd9" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:36.101243 5430 raft_consensus.cc:3037] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.101315 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f27a79cf3a044383918f5d6f600a0fd9" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:36.101400 5827 raft_consensus.cc:3037] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.101851 5561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f27a79cf3a044383918f5d6f600a0fd9" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:36.101897 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f27a79cf3a044383918f5d6f600a0fd9" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:36.101948 5561 raft_consensus.cc:3037] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.101989 6096 raft_consensus.cc:3037] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.102157 5430 raft_consensus.cc:2454] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.102444 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f27a79cf3a044383918f5d6f600a0fd9" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:36.102530 5960 raft_consensus.cc:3037] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.102571 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f27a79cf3a044383918f5d6f600a0fd9" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:36.102674 5296 raft_consensus.cc:3037] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.103017 5561 raft_consensus.cc:2454] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.103372 5960 raft_consensus.cc:2454] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.103623 5296 raft_consensus.cc:2454] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.104228 6096 raft_consensus.cc:2454] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.104230 5628 leader_election.cc:302] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, 9782508a47ba4403a39ca49fff7677fa, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:36.105829 6711 raft_consensus.cc:2781] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:36.105921 6711 raft_consensus.cc:686] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Becoming Leader. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Running, Role: LEADER
I20240501 19:57:36.106063 6711 consensus_queue.cc:237] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.108027 5153 catalog_manager.cc:5484] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d4855c096ac4e1e936abf15834332ae (127.20.210.4). New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:36.110544 6977 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:36.110625 6977 ts_tablet_manager.cc:1367] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20240501 19:57:36.110946 5827 raft_consensus.cc:2454] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.111057 6977 raft_consensus.cc:348] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.111164 6977 raft_consensus.cc:374] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.111210 6977 raft_consensus.cc:729] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.111351 6977 consensus_queue.cc:260] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.111577 6977 ts_tablet_manager.cc:1398] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.111828 6971 raft_consensus.cc:482] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:36.111914 6971 raft_consensus.cc:504] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.112438 6971 leader_election.cc:288] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:36.112813 5561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a84349ba37496289351ac689f77eab" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:36.112864 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a84349ba37496289351ac689f77eab" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:36.112927 5561 raft_consensus.cc:2454] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:36.112965 5696 raft_consensus.cc:2454] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:36.113118 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a84349ba37496289351ac689f77eab" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:36.113226 5960 raft_consensus.cc:2454] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:36.113492 5764 leader_election.cc:302] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:36.113649 6971 raft_consensus.cc:2781] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:36.113688 6971 raft_consensus.cc:482] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:36.113729 6971 raft_consensus.cc:3037] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.113859 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a84349ba37496289351ac689f77eab" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:36.113922 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a84349ba37496289351ac689f77eab" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:36.113958 5296 raft_consensus.cc:2454] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:36.114012 5430 raft_consensus.cc:2454] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:36.114351 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a84349ba37496289351ac689f77eab" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:36.114459 6096 raft_consensus.cc:2454] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 0.
I20240501 19:57:36.114583 6971 raft_consensus.cc:504] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.115077 6971 leader_election.cc:288] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [CANDIDATE]: Term 1 election: Requested vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:36.115435 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a84349ba37496289351ac689f77eab" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:36.115432 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a84349ba37496289351ac689f77eab" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:36.115523 5296 raft_consensus.cc:3037] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.115527 5430 raft_consensus.cc:3037] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.115715 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a84349ba37496289351ac689f77eab" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:36.115795 5960 raft_consensus.cc:3037] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.116253 5296 raft_consensus.cc:2454] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:36.116364 5430 raft_consensus.cc:2454] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:36.116598 5960 raft_consensus.cc:2454] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:36.116817 5561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a84349ba37496289351ac689f77eab" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:36.116860 5764 leader_election.cc:302] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 9544b0e566c54a62a6135f3bff47c4f2, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:36.116905 5561 raft_consensus.cc:3037] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.117048 6971 raft_consensus.cc:2781] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:36.117120 6971 raft_consensus.cc:686] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Becoming Leader. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Running, Role: LEADER
I20240501 19:57:36.117226 6971 consensus_queue.cc:237] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.117548 5561 raft_consensus.cc:2454] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:36.117848 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a84349ba37496289351ac689f77eab" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:36.117874 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8a84349ba37496289351ac689f77eab" candidate_uuid: "1865147cd5a346ad8549f18fa8cc26c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:36.117937 5696 raft_consensus.cc:3037] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.117961 6096 raft_consensus.cc:3037] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.118727 6096 raft_consensus.cc:2454] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:36.118731 5696 raft_consensus.cc:2454] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1865147cd5a346ad8549f18fa8cc26c3 in term 1.
I20240501 19:57:36.119354 5153 catalog_manager.cc:5484] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5). New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:36.249028 7003 raft_consensus.cc:482] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:36.249128 7003 raft_consensus.cc:504] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.249857 7003 leader_election.cc:288] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:36.249873 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f698b376e407da78c7d439553896a" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:36.250033 5296 raft_consensus.cc:2454] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.250068 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f698b376e407da78c7d439553896a" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:36.250200 5430 raft_consensus.cc:2454] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.250245 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f698b376e407da78c7d439553896a" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:36.250402 5827 raft_consensus.cc:2454] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.250459 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f698b376e407da78c7d439553896a" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:36.250504 5561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f698b376e407da78c7d439553896a" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:36.250583 5960 raft_consensus.cc:2454] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.250561 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f698b376e407da78c7d439553896a" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:36.250608 5561 raft_consensus.cc:2454] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.250689 6096 raft_consensus.cc:2454] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.250798 5628 leader_election.cc:302] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:36.250980 7003 raft_consensus.cc:2781] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:36.251048 7003 raft_consensus.cc:482] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:36.251089 7003 raft_consensus.cc:3037] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.251870 7003 raft_consensus.cc:504] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.252329 7003 leader_election.cc:288] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 election: Requested vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:36.252454 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f698b376e407da78c7d439553896a" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:36.252545 5296 raft_consensus.cc:3037] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.252528 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f698b376e407da78c7d439553896a" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:36.252625 5430 raft_consensus.cc:3037] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.252640 5561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f698b376e407da78c7d439553896a" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:36.252733 5561 raft_consensus.cc:3037] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.253001 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f698b376e407da78c7d439553896a" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:36.253091 6096 raft_consensus.cc:3037] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.253412 5296 raft_consensus.cc:2454] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.253520 5430 raft_consensus.cc:2454] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.253620 5561 raft_consensus.cc:2454] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.253716 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f698b376e407da78c7d439553896a" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:36.253821 5827 raft_consensus.cc:3037] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.254031 6096 raft_consensus.cc:2454] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.254014 5631 leader_election.cc:302] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 4d4855c096ac4e1e936abf15834332ae, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:36.254098 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f698b376e407da78c7d439553896a" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:36.254173 7003 raft_consensus.cc:2781] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:36.254200 5960 raft_consensus.cc:3037] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.254499 7003 raft_consensus.cc:686] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Becoming Leader. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Running, Role: LEADER
I20240501 19:57:36.254631 5827 raft_consensus.cc:2454] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.254630 7003 consensus_queue.cc:237] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.255087 5960 raft_consensus.cc:2454] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.256651 5153 catalog_manager.cc:5484] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d4855c096ac4e1e936abf15834332ae (127.20.210.4). New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } }
W20240501 19:57:36.361600 7007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:36.361805 7007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:36.364564 7007 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240501 19:57:36.428746 5296 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.428747 5430 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.429123 7003 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.429736 5960 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.429831 5561 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.430126 6711 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.430799 6096 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.430954 5827 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.431458 6711 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.434876 7003 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.435194 7003 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.435513 7003 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.436380 5297 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.436990 5561 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.437297 6096 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.437333 5428 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.437407 6972 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.437601 6971 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.437726 6974 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.437865 5960 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.442510 5696 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.443413 6973 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.443861 5297 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.444618 5428 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.444684 5962 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.445541 6097 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.446545 6974 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.446847 6974 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.447211 5563 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.447497 6711 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.454555 6711 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.455449 6711 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.455689 6711 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.455981 6711 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.458395 5828 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.465371 7003 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.506341 6934 mvcc.cc:205] Tried to move back new op lower bound from 7022974797557411840 to 7022974796258430976. Current Snapshot: MvccSnapshot[applied={T|T < 7022974797557411840}]
I20240501 19:57:36.514415 6908 mvcc.cc:205] Tried to move back new op lower bound from 7022974797544542208 to 7022974796213747712. Current Snapshot: MvccSnapshot[applied={T|T < 7022974797544542208}]
I20240501 19:57:36.521948 6924 mvcc.cc:205] Tried to move back new op lower bound from 7022974797583794176 to 7022974796821848064. Current Snapshot: MvccSnapshot[applied={T|T < 7022974797583794176 or (T in {7022974797583794176})}]
I20240501 19:57:36.524896 6917 mvcc.cc:205] Tried to move back new op lower bound from 7022974797544542208 to 7022974796213747712. Current Snapshot: MvccSnapshot[applied={T|T < 7022974797528023040}]
I20240501 19:57:36.533350 7059 mvcc.cc:205] Tried to move back new op lower bound from 7022974797590302720 to 7022974796258430976. Current Snapshot: MvccSnapshot[applied={T|T < 7022974797557411840}]
INSERT report
rows total: 2000
time total: 151.136 ms
time per row: 0.075568 ms
I20240501 19:57:36.542343 6927 mvcc.cc:205] Tried to move back new op lower bound from 7022974797528023040 to 7022974796213747712. Current Snapshot: MvccSnapshot[applied={T|T < 7022974797528023040 or (T in {7022974797528023040})}]
I20240501 19:57:36.567380 5154 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47970:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 19:57:36.568014 5154 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_4 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 19:57:36.574841 5808 tablet_service.cc:1467] Processing CreateTablet for tablet 9ecbb3dec08047478ded9809f3a0b859 (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.575111 5808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ecbb3dec08047478ded9809f3a0b859. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.577103 6977 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:36.577139 5807 tablet_service.cc:1467] Processing CreateTablet for tablet c59efbd84dd847188714271f4ba09ecb (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.577369 5807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c59efbd84dd847188714271f4ba09ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.577682 5809 tablet_service.cc:1467] Processing CreateTablet for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.577909 5809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.577944 6977 tablet_bootstrap.cc:654] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.578015 6077 tablet_service.cc:1467] Processing CreateTablet for tablet 9ecbb3dec08047478ded9809f3a0b859 (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.578305 6077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ecbb3dec08047478ded9809f3a0b859. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.579173 6977 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:36.579247 6977 ts_tablet_manager.cc:1367] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240501 19:57:36.579633 6977 raft_consensus.cc:348] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.579730 6977 raft_consensus.cc:374] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.579680 6076 tablet_service.cc:1467] Processing CreateTablet for tablet c59efbd84dd847188714271f4ba09ecb (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.579767 6977 raft_consensus.cc:729] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.579893 6076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c59efbd84dd847188714271f4ba09ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.579875 6977 consensus_queue.cc:260] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.580173 6977 ts_tablet_manager.cc:1398] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:36.580307 6990 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:36.580322 6977 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:36.581048 6075 tablet_service.cc:1467] Processing CreateTablet for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.581112 6990 tablet_bootstrap.cc:654] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.581112 6977 tablet_bootstrap.cc:654] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.581280 6075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.582180 5542 tablet_service.cc:1467] Processing CreateTablet for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.582271 5541 tablet_service.cc:1467] Processing CreateTablet for tablet c59efbd84dd847188714271f4ba09ecb (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.582463 5542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.583235 5277 tablet_service.cc:1467] Processing CreateTablet for tablet c59efbd84dd847188714271f4ba09ecb (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.583411 5275 tablet_service.cc:1467] Processing CreateTablet for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.583437 5409 tablet_service.cc:1467] Processing CreateTablet for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.583479 5277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c59efbd84dd847188714271f4ba09ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.583614 5275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.583617 5409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.583256 5276 tablet_service.cc:1467] Processing CreateTablet for tablet 9ecbb3dec08047478ded9809f3a0b859 (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.584422 5276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ecbb3dec08047478ded9809f3a0b859. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.582180 5543 tablet_service.cc:1467] Processing CreateTablet for tablet 9ecbb3dec08047478ded9809f3a0b859 (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.584800 5543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ecbb3dec08047478ded9809f3a0b859. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.585741 5408 tablet_service.cc:1467] Processing CreateTablet for tablet 9ecbb3dec08047478ded9809f3a0b859 (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.585947 5408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ecbb3dec08047478ded9809f3a0b859. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.586174 5941 tablet_service.cc:1467] Processing CreateTablet for tablet c59efbd84dd847188714271f4ba09ecb (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.586421 5941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c59efbd84dd847188714271f4ba09ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.586495 5410 tablet_service.cc:1467] Processing CreateTablet for tablet c59efbd84dd847188714271f4ba09ecb (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.586673 5410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c59efbd84dd847188714271f4ba09ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.586892 7086 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:36.587728 7086 tablet_bootstrap.cc:654] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.588238 7085 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:36.588238 6987 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:36.588411 5940 tablet_service.cc:1467] Processing CreateTablet for tablet 9ecbb3dec08047478ded9809f3a0b859 (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.588650 5940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ecbb3dec08047478ded9809f3a0b859. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.588660 5541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c59efbd84dd847188714271f4ba09ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.588981 6987 tablet_bootstrap.cc:654] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.589015 7085 tablet_bootstrap.cc:654] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.589519 7084 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:36.590481 7084 tablet_bootstrap.cc:654] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.590577 5942 tablet_service.cc:1467] Processing CreateTablet for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.590797 5942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.591150 5674 tablet_service.cc:1467] Processing CreateTablet for tablet 9ecbb3dec08047478ded9809f3a0b859 (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.591385 5674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ecbb3dec08047478ded9809f3a0b859. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.591779 7086 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:36.591997 7086 ts_tablet_manager.cc:1367] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 19:57:36.592393 7086 raft_consensus.cc:348] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.592506 7086 raft_consensus.cc:374] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.592537 7086 raft_consensus.cc:729] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.592681 5673 tablet_service.cc:1467] Processing CreateTablet for tablet c59efbd84dd847188714271f4ba09ecb (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.592868 5673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c59efbd84dd847188714271f4ba09ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.592674 7086 consensus_queue.cc:260] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.593241 6990 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:36.593327 6990 ts_tablet_manager.cc:1367] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20240501 19:57:36.593431 5675 tablet_service.cc:1467] Processing CreateTablet for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b (DEFAULT_TABLE table=rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.593626 5675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:36.593709 6990 raft_consensus.cc:348] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.593811 6990 raft_consensus.cc:374] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.593843 6990 raft_consensus.cc:729] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.593942 6990 consensus_queue.cc:260] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.594300 6990 ts_tablet_manager.cc:1398] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.594460 6990 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:36.594866 7086 ts_tablet_manager.cc:1398] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:57:36.595300 6990 tablet_bootstrap.cc:654] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.595402 6989 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:36.596127 6989 tablet_bootstrap.cc:654] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.596349 7086 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:36.597959 7086 tablet_bootstrap.cc:654] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.598366 7085 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:36.598455 7085 ts_tablet_manager.cc:1367] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240501 19:57:36.598760 7084 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:36.598838 7084 ts_tablet_manager.cc:1367] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240501 19:57:36.598863 6977 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:36.598863 6990 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:36.598929 6990 ts_tablet_manager.cc:1367] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 19:57:36.598929 6977 ts_tablet_manager.cc:1367] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.001s
I20240501 19:57:36.598872 7085 raft_consensus.cc:348] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.598973 7085 raft_consensus.cc:374] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.599000 7085 raft_consensus.cc:729] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.599112 7085 consensus_queue.cc:260] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.599246 7084 raft_consensus.cc:348] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.599354 7084 raft_consensus.cc:374] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.599282 6990 raft_consensus.cc:348] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.599383 7084 raft_consensus.cc:729] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.599323 6977 raft_consensus.cc:348] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.599401 6990 raft_consensus.cc:374] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.599431 6977 raft_consensus.cc:374] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.599433 6990 raft_consensus.cc:729] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.599450 7085 ts_tablet_manager.cc:1398] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:36.599462 6977 raft_consensus.cc:729] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.599491 7084 consensus_queue.cc:260] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.599575 7085 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:36.599545 6990 consensus_queue.cc:260] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.599557 6977 consensus_queue.cc:260] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.599730 7084 ts_tablet_manager.cc:1398] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:36.599802 6990 ts_tablet_manager.cc:1398] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.599820 6977 ts_tablet_manager.cc:1398] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:36.599872 7084 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:36.599952 6977 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:36.599958 6990 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:36.600780 7084 tablet_bootstrap.cc:654] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.600813 6977 tablet_bootstrap.cc:654] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.600847 7085 tablet_bootstrap.cc:654] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.601815 6989 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:36.601903 6989 ts_tablet_manager.cc:1367] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240501 19:57:36.602325 6989 raft_consensus.cc:348] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.602470 6989 raft_consensus.cc:374] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.602494 6990 tablet_bootstrap.cc:654] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.602512 6989 raft_consensus.cc:729] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.602633 6989 consensus_queue.cc:260] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.602921 6989 ts_tablet_manager.cc:1398] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.603083 6989 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:36.603911 6989 tablet_bootstrap.cc:654] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.604245 7086 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:36.604355 7086 ts_tablet_manager.cc:1367] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240501 19:57:36.604487 6990 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:36.604584 6990 ts_tablet_manager.cc:1367] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240501 19:57:36.604903 7086 raft_consensus.cc:348] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.605007 6990 raft_consensus.cc:348] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.605077 7086 raft_consensus.cc:374] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.605139 6990 raft_consensus.cc:374] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.605152 7086 raft_consensus.cc:729] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.605185 6990 raft_consensus.cc:729] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.605320 6990 consensus_queue.cc:260] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.605430 7084 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:36.605345 7086 consensus_queue.cc:260] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.605527 7084 ts_tablet_manager.cc:1367] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240501 19:57:36.605633 6990 ts_tablet_manager.cc:1398] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.605914 7084 raft_consensus.cc:348] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.606025 7084 raft_consensus.cc:374] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.606065 7084 raft_consensus.cc:729] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.606088 7086 ts_tablet_manager.cc:1398] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:36.606189 7084 consensus_queue.cc:260] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.606374 7086 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:36.606523 7084 ts_tablet_manager.cc:1398] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.606704 7084 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:36.606724 6977 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:36.606829 6977 ts_tablet_manager.cc:1367] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240501 19:57:36.607309 7086 tablet_bootstrap.cc:654] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.607592 7084 tablet_bootstrap.cc:654] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.607908 6977 raft_consensus.cc:348] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.608017 6977 raft_consensus.cc:374] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.608091 6977 raft_consensus.cc:729] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.608225 6977 consensus_queue.cc:260] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.608461 7085 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:36.608539 7085 ts_tablet_manager.cc:1367] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240501 19:57:36.608990 7085 raft_consensus.cc:348] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.609118 7085 raft_consensus.cc:374] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.609158 7085 raft_consensus.cc:729] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.609287 7085 consensus_queue.cc:260] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.609550 7085 ts_tablet_manager.cc:1398] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:36.609707 7085 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:36.610200 7086 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:36.610366 7086 ts_tablet_manager.cc:1367] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240501 19:57:36.610567 7085 tablet_bootstrap.cc:654] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.610872 7086 raft_consensus.cc:348] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.611017 7086 raft_consensus.cc:374] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.611075 7086 raft_consensus.cc:729] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.611253 7086 consensus_queue.cc:260] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.611512 7084 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:36.611536 7086 ts_tablet_manager.cc:1398] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.611596 7084 ts_tablet_manager.cc:1367] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240501 19:57:36.612118 6977 ts_tablet_manager.cc:1398] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20240501 19:57:36.612211 6989 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:36.612313 6989 ts_tablet_manager.cc:1367] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240501 19:57:36.612747 7085 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:36.612709 6989 raft_consensus.cc:348] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.612812 6989 raft_consensus.cc:374] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.612831 7085 ts_tablet_manager.cc:1367] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:57:36.612843 6989 raft_consensus.cc:729] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.612954 6989 consensus_queue.cc:260] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.613209 6989 ts_tablet_manager.cc:1398] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.613191 7085 raft_consensus.cc:348] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.613349 6989 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:36.613528 7084 raft_consensus.cc:348] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.613623 7084 raft_consensus.cc:374] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.613659 7084 raft_consensus.cc:729] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.613775 7084 consensus_queue.cc:260] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.614053 7084 ts_tablet_manager.cc:1398] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:36.613287 7085 raft_consensus.cc:374] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.614230 7085 raft_consensus.cc:729] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.614354 7085 consensus_queue.cc:260] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.614497 6989 tablet_bootstrap.cc:654] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.614666 7085 ts_tablet_manager.cc:1398] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:36.615864 6989 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:36.615949 6989 ts_tablet_manager.cc:1367] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:36.616364 6989 raft_consensus.cc:348] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.616489 6989 raft_consensus.cc:374] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.616523 6989 raft_consensus.cc:729] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.616647 6989 consensus_queue.cc:260] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.617686 6989 ts_tablet_manager.cc:1398] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:36.618543 6987 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:36.618628 6987 ts_tablet_manager.cc:1367] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.000s
I20240501 19:57:36.619050 6987 raft_consensus.cc:348] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.619187 6987 raft_consensus.cc:374] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.619220 6987 raft_consensus.cc:729] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.619349 6987 consensus_queue.cc:260] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.619623 6987 ts_tablet_manager.cc:1398] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:36.619829 6987 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:36.620651 6987 tablet_bootstrap.cc:654] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.621901 6987 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:36.621960 6987 ts_tablet_manager.cc:1367] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:36.622234 6987 raft_consensus.cc:348] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.622349 6987 raft_consensus.cc:374] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.622370 6987 raft_consensus.cc:729] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.622489 6987 consensus_queue.cc:260] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.622732 6987 ts_tablet_manager.cc:1398] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:36.622951 6987 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:36.623590 6987 tablet_bootstrap.cc:654] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:36.624584 6987 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:36.624676 6987 ts_tablet_manager.cc:1367] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:57:36.625101 6987 raft_consensus.cc:348] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.625187 6987 raft_consensus.cc:374] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:36.625217 6987 raft_consensus.cc:729] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:36.625329 6987 consensus_queue.cc:260] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.625536 6987 ts_tablet_manager.cc:1398] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:36.627503 6936 raft_consensus.cc:482] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:36.627596 6936 raft_consensus.cc:504] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.628113 6936 leader_election.cc:288] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
I20240501 19:57:36.628228 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:36.628353 5828 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:36.628495 5297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:36.628589 5297 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:36.628806 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:36.628912 6096 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:36.629086 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:36.629134 5499 leader_election.cc:302] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, a881063bd86945a8871c7c102d8235c8, ad52038f1d38489ea464a8fbc72daac0, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:36.629192 5430 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:36.629288 6936 raft_consensus.cc:2781] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:36.629343 6936 raft_consensus.cc:482] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:36.629385 6936 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.629390 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:36.629488 5962 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:36.629663 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:36.629773 5696 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:36.630352 6936 raft_consensus.cc:504] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.630897 6936 leader_election.cc:288] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Requested vote from peers 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
I20240501 19:57:36.630977 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:36.631117 5828 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.631142 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:36.631251 6096 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.631917 6096 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:36.631924 5828 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:36.632095 5297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:36.632189 5297 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.632855 5297 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:36.633064 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:36.633175 5962 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.633869 5962 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:36.634059 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:36.634106 5499 leader_election.cc:302] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 9544b0e566c54a62a6135f3bff47c4f2, a881063bd86945a8871c7c102d8235c8, ad52038f1d38489ea464a8fbc72daac0, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:36.634188 5430 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.634315 6936 raft_consensus.cc:2781] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:36.634335 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:36.634425 5696 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.634466 6936 raft_consensus.cc:686] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Becoming Leader. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Running, Role: LEADER
I20240501 19:57:36.634639 6936 consensus_queue.cc:237] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.635223 5696 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:36.636233 5430 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:36.638623 6910 raft_consensus.cc:482] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:36.638727 6910 raft_consensus.cc:504] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.638654 5147 catalog_manager.cc:5484] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:36.640244 6910 leader_election.cc:288] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:36.646409 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59efbd84dd847188714271f4ba09ecb" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:36.646543 5696 raft_consensus.cc:2454] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:36.646842 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59efbd84dd847188714271f4ba09ecb" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:36.646950 5828 raft_consensus.cc:2454] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:36.648707 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59efbd84dd847188714271f4ba09ecb" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:36.648818 5562 raft_consensus.cc:2454] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:36.649080 5898 leader_election.cc:302] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:36.649281 6910 raft_consensus.cc:2781] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:36.649328 6910 raft_consensus.cc:482] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:36.649361 6910 raft_consensus.cc:3037] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.649924 6910 raft_consensus.cc:504] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.650367 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59efbd84dd847188714271f4ba09ecb" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:36.650466 5430 raft_consensus.cc:2454] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:36.650529 6910 leader_election.cc:288] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [CANDIDATE]: Term 1 election: Requested vote from peers 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:36.650789 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59efbd84dd847188714271f4ba09ecb" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:36.650862 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59efbd84dd847188714271f4ba09ecb" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:36.650897 5696 raft_consensus.cc:3037] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.650961 5828 raft_consensus.cc:3037] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.651748 5696 raft_consensus.cc:2454] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:36.651780 5828 raft_consensus.cc:2454] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:36.652312 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59efbd84dd847188714271f4ba09ecb" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:36.652429 5430 raft_consensus.cc:3037] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.652699 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59efbd84dd847188714271f4ba09ecb" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:36.652788 5562 raft_consensus.cc:3037] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.653208 5430 raft_consensus.cc:2454] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:36.653545 5562 raft_consensus.cc:2454] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:36.653792 5898 leader_election.cc:302] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:36.653993 6910 raft_consensus.cc:2781] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:36.654168 6910 raft_consensus.cc:686] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Becoming Leader. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Running, Role: LEADER
I20240501 19:57:36.654454 6910 consensus_queue.cc:237] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:36.655781 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59efbd84dd847188714271f4ba09ecb" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:36.655897 6096 raft_consensus.cc:2454] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:36.656047 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59efbd84dd847188714271f4ba09ecb" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:36.656121 6097 raft_consensus.cc:3037] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.656940 6097 raft_consensus.cc:2454] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:36.656831 5147 catalog_manager.cc:5484] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6). New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:36.657852 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59efbd84dd847188714271f4ba09ecb" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:36.657960 5296 raft_consensus.cc:3037] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.657863 5297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c59efbd84dd847188714271f4ba09ecb" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:36.659068 5296 raft_consensus.cc:2454] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:36.660605 6936 raft_consensus.cc:482] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:36.660696 6936 raft_consensus.cc:504] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.661206 6936 leader_election.cc:288] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:36.661403 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:36.661509 5430 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:36.661512 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:36.661597 5296 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:36.661758 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:36.661837 5828 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:36.661947 7066 raft_consensus.cc:482] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:36.662027 7066 raft_consensus.cc:504] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.662073 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:36.662163 5696 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:36.662565 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:36.662600 7066 leader_election.cc:288] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:36.662671 5962 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:36.662832 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:36.662914 5296 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.663030 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:36.663030 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:36.663130 5430 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.663130 5828 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.663408 5499 leader_election.cc:302] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:36.663450 5629 leader_election.cc:302] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:36.663559 6936 raft_consensus.cc:2781] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:36.663609 6936 raft_consensus.cc:482] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:36.663650 6936 raft_consensus.cc:3037] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.663985 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:36.664002 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:36.664081 5962 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.664461 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:36.664559 6096 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:36.664558 7066 raft_consensus.cc:2781] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:36.664613 7066 raft_consensus.cc:482] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:36.664649 7066 raft_consensus.cc:3037] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.664588 6936 raft_consensus.cc:504] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.665191 6936 leader_election.cc:288] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Requested vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:36.665338 5562 raft_consensus.cc:2379] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4d4855c096ac4e1e936abf15834332ae in current term 1: Already voted for candidate f5287d5ae4d84bd692b0f7985cd94171 in this term.
I20240501 19:57:36.665449 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:36.665541 6096 raft_consensus.cc:3037] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.665536 7066 raft_consensus.cc:504] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.666010 7066 leader_election.cc:288] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 election: Requested vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:36.666352 6096 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:36.666388 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:36.666420 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:36.666482 5828 raft_consensus.cc:3037] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.666523 5562 raft_consensus.cc:2379] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4d4855c096ac4e1e936abf15834332ae in current term 1: Already voted for candidate f5287d5ae4d84bd692b0f7985cd94171 in this term.
I20240501 19:57:36.666586 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:36.666664 5960 raft_consensus.cc:3037] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.667249 5828 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:36.667438 5960 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:36.667475 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:36.667563 5296 raft_consensus.cc:3037] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.668150 5499 leader_election.cc:302] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 9544b0e566c54a62a6135f3bff47c4f2, a881063bd86945a8871c7c102d8235c8, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:36.668365 5296 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.668566 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:36.668663 5430 raft_consensus.cc:3037] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:36.668654 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:36.668787 5828 raft_consensus.cc:2379] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4d4855c096ac4e1e936abf15834332ae in current term 1: Already voted for candidate f5287d5ae4d84bd692b0f7985cd94171 in this term.
I20240501 19:57:36.669414 5430 raft_consensus.cc:2454] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:36.669724 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:36.669855 5962 raft_consensus.cc:2379] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4d4855c096ac4e1e936abf15834332ae in current term 1: Already voted for candidate f5287d5ae4d84bd692b0f7985cd94171 in this term.
I20240501 19:57:36.670367 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:36.670462 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:36.670511 6096 raft_consensus.cc:2379] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4d4855c096ac4e1e936abf15834332ae in current term 1: Already voted for candidate f5287d5ae4d84bd692b0f7985cd94171 in this term.
I20240501 19:57:36.670593 5296 raft_consensus.cc:2379] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f5287d5ae4d84bd692b0f7985cd94171 in current term 1: Already voted for candidate 4d4855c096ac4e1e936abf15834332ae in this term.
I20240501 19:57:36.670594 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:36.670708 5430 raft_consensus.cc:2379] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f5287d5ae4d84bd692b0f7985cd94171 in current term 1: Already voted for candidate 4d4855c096ac4e1e936abf15834332ae in this term.
I20240501 19:57:36.670689 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:36.670796 6097 raft_consensus.cc:2362] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1. Re-sending same reply.
I20240501 19:57:36.671018 5631 leader_election.cc:302] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 4d4855c096ac4e1e936abf15834332ae, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0; no voters: 1865147cd5a346ad8549f18fa8cc26c3, 9544b0e566c54a62a6135f3bff47c4f2, a881063bd86945a8871c7c102d8235c8, f5287d5ae4d84bd692b0f7985cd94171
I20240501 19:57:36.671190 7066 raft_consensus.cc:2726] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240501 19:57:36.671242 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ecbb3dec08047478ded9809f3a0b859" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:36.671394 5696 raft_consensus.cc:2379] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f5287d5ae4d84bd692b0f7985cd94171 in current term 1: Already voted for candidate 4d4855c096ac4e1e936abf15834332ae in this term.
I20240501 19:57:36.671541 6936 raft_consensus.cc:2781] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:36.671617 6936 raft_consensus.cc:686] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Becoming Leader. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Running, Role: LEADER
I20240501 19:57:36.671734 6936 consensus_queue.cc:237] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:36.673302 5147 catalog_manager.cc:5484] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } }
W20240501 19:57:36.789373 7120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:36.789546 7120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:36.792282 7120 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240501 19:57:36.858407 5430 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.859236 6936 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.862475 5827 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.862766 6097 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.863027 6096 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.863317 5696 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.863626 5694 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.863818 5828 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.864380 5962 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.864712 5960 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.864897 6936 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.865186 6936 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.865440 6936 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.865486 5296 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.865677 7105 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.865921 5297 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.865913 7105 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.866289 7105 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.866519 5428 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.866577 7105 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.874595 6936 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.875047 6936 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.875334 6936 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.876116 6936 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.899050 6923 mvcc.cc:205] Tried to move back new op lower bound from 7022974799284514816 to 7022974798529523712. Current Snapshot: MvccSnapshot[applied={T|T < 7022974799284514816}]
I20240501 19:57:36.902918 5829 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.903250 6910 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.905189 5296 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.905561 7117 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.911610 5696 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.911891 5428 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.911998 7117 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.912272 5562 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.912660 6910 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.912941 6910 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:36.913785 6097 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:36.914160 6910 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 146.08 ms
time per row: 0.0730399 ms
I20240501 19:57:36.994269 5152 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47970:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 19:57:36.994777 5152 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_5 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 19:57:36.999826 5408 tablet_service.cc:1467] Processing CreateTablet for tablet d6b6181bae544db0b6bbdc6bcd01a40d (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.999882 5277 tablet_service.cc:1467] Processing CreateTablet for tablet d6b6181bae544db0b6bbdc6bcd01a40d (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:36.999985 5541 tablet_service.cc:1467] Processing CreateTablet for tablet d6b6181bae544db0b6bbdc6bcd01a40d (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.000108 5408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b6181bae544db0b6bbdc6bcd01a40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.000140 5277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b6181bae544db0b6bbdc6bcd01a40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.000226 5541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b6181bae544db0b6bbdc6bcd01a40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.002410 7084 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:37.002770 5277 tablet_service.cc:1467] Processing CreateTablet for tablet f0df1e5e85294f60b8de22053dd577f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.002811 5276 tablet_service.cc:1467] Processing CreateTablet for tablet 1211024f45e142979fc1e5e8fff010ce (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.002980 5277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0df1e5e85294f60b8de22053dd577f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.003187 7084 tablet_bootstrap.cc:654] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.003379 5276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1211024f45e142979fc1e5e8fff010ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.003655 7085 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:37.004443 7085 tablet_bootstrap.cc:654] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.005000 7086 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:37.005808 7086 tablet_bootstrap.cc:654] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.007170 5809 tablet_service.cc:1467] Processing CreateTablet for tablet d6b6181bae544db0b6bbdc6bcd01a40d (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.007402 5809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b6181bae544db0b6bbdc6bcd01a40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.008811 6977 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:37.008952 5807 tablet_service.cc:1467] Processing CreateTablet for tablet f0df1e5e85294f60b8de22053dd577f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.009105 5807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0df1e5e85294f60b8de22053dd577f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.009495 6977 tablet_bootstrap.cc:654] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.010437 6977 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:37.010509 6977 ts_tablet_manager.cc:1367] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:57:37.010756 5808 tablet_service.cc:1467] Processing CreateTablet for tablet 1211024f45e142979fc1e5e8fff010ce (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.010924 5808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1211024f45e142979fc1e5e8fff010ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.010886 6977 raft_consensus.cc:348] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.011004 6977 raft_consensus.cc:374] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.011035 6977 raft_consensus.cc:729] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.011147 6977 consensus_queue.cc:260] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.011404 6977 ts_tablet_manager.cc:1398] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.011519 6977 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:37.012164 6977 tablet_bootstrap.cc:654] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.013298 6075 tablet_service.cc:1467] Processing CreateTablet for tablet d6b6181bae544db0b6bbdc6bcd01a40d (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.013347 6076 tablet_service.cc:1467] Processing CreateTablet for tablet f0df1e5e85294f60b8de22053dd577f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.013535 6075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b6181bae544db0b6bbdc6bcd01a40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.013496 5675 tablet_service.cc:1467] Processing CreateTablet for tablet d6b6181bae544db0b6bbdc6bcd01a40d (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.013698 6076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0df1e5e85294f60b8de22053dd577f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.013716 5675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b6181bae544db0b6bbdc6bcd01a40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.014480 5673 tablet_service.cc:1467] Processing CreateTablet for tablet f0df1e5e85294f60b8de22053dd577f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.014657 5673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0df1e5e85294f60b8de22053dd577f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.015461 6990 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:37.016093 6990 tablet_bootstrap.cc:654] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.016376 6989 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:37.016521 5674 tablet_service.cc:1467] Processing CreateTablet for tablet 1211024f45e142979fc1e5e8fff010ce (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.016716 5674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1211024f45e142979fc1e5e8fff010ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.017098 6989 tablet_bootstrap.cc:654] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.017971 6989 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:37.018060 6989 ts_tablet_manager.cc:1367] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:37.018446 6989 raft_consensus.cc:348] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.018558 6989 raft_consensus.cc:374] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.018602 6989 raft_consensus.cc:729] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.018563 6077 tablet_service.cc:1467] Processing CreateTablet for tablet 1211024f45e142979fc1e5e8fff010ce (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.018754 6077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1211024f45e142979fc1e5e8fff010ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.018707 6989 consensus_queue.cc:260] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.018985 6989 ts_tablet_manager.cc:1398] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.019120 6989 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:37.019928 6989 tablet_bootstrap.cc:654] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.020865 6989 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:37.020984 6989 ts_tablet_manager.cc:1367] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:37.021382 6989 raft_consensus.cc:348] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.021528 6989 raft_consensus.cc:374] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.021600 6989 raft_consensus.cc:729] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.021703 6989 consensus_queue.cc:260] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.021720 5942 tablet_service.cc:1467] Processing CreateTablet for tablet d6b6181bae544db0b6bbdc6bcd01a40d (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.021927 5942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b6181bae544db0b6bbdc6bcd01a40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.021935 6989 ts_tablet_manager.cc:1398] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.022490 5941 tablet_service.cc:1467] Processing CreateTablet for tablet 1211024f45e142979fc1e5e8fff010ce (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.022681 5941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1211024f45e142979fc1e5e8fff010ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.023684 6987 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:37.024395 6987 tablet_bootstrap.cc:654] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.024408 5940 tablet_service.cc:1467] Processing CreateTablet for tablet f0df1e5e85294f60b8de22053dd577f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.024658 5940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0df1e5e85294f60b8de22053dd577f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.025626 6987 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:37.025709 6987 ts_tablet_manager.cc:1367] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240501 19:57:37.026021 6987 raft_consensus.cc:348] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.026132 6987 raft_consensus.cc:374] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.026191 6987 raft_consensus.cc:729] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.026329 6987 consensus_queue.cc:260] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.026613 6987 ts_tablet_manager.cc:1398] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.026760 6987 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:37.027200 5408 tablet_service.cc:1467] Processing CreateTablet for tablet f0df1e5e85294f60b8de22053dd577f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.027433 6987 tablet_bootstrap.cc:654] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.027432 5408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0df1e5e85294f60b8de22053dd577f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.028100 5410 tablet_service.cc:1467] Processing CreateTablet for tablet 1211024f45e142979fc1e5e8fff010ce (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.028297 5410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1211024f45e142979fc1e5e8fff010ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.028617 5543 tablet_service.cc:1467] Processing CreateTablet for tablet 1211024f45e142979fc1e5e8fff010ce (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.028846 5543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1211024f45e142979fc1e5e8fff010ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.033216 6989 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:37.033263 6987 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:37.033346 6987 ts_tablet_manager.cc:1367] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240501 19:57:37.033644 7085 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:37.033651 6987 raft_consensus.cc:348] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.033717 7085 ts_tablet_manager.cc:1367] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.000s
I20240501 19:57:37.033761 6987 raft_consensus.cc:374] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.033790 6987 raft_consensus.cc:729] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.033885 6987 consensus_queue.cc:260] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.033984 6989 tablet_bootstrap.cc:654] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.034149 6987 ts_tablet_manager.cc:1398] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.034175 7085 raft_consensus.cc:348] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.034309 7085 raft_consensus.cc:374] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.034318 6987 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:37.034341 7085 raft_consensus.cc:729] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.034479 7085 consensus_queue.cc:260] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.034756 7085 ts_tablet_manager.cc:1398] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.034893 7085 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:37.035292 6987 tablet_bootstrap.cc:654] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.035431 5541 tablet_service.cc:1467] Processing CreateTablet for tablet f0df1e5e85294f60b8de22053dd577f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.035558 7085 tablet_bootstrap.cc:654] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.035650 5541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0df1e5e85294f60b8de22053dd577f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.037254 6987 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:37.037264 7084 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:37.037317 6987 ts_tablet_manager.cc:1367] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:57:37.037340 7084 ts_tablet_manager.cc:1367] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.001s
I20240501 19:57:37.037708 6987 raft_consensus.cc:348] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.037731 7084 raft_consensus.cc:348] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.037799 6987 raft_consensus.cc:374] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.037837 6987 raft_consensus.cc:729] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.037837 7084 raft_consensus.cc:374] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.037868 7084 raft_consensus.cc:729] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.037948 6987 consensus_queue.cc:260] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.037976 7084 consensus_queue.cc:260] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.038195 6987 ts_tablet_manager.cc:1398] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.038241 7084 ts_tablet_manager.cc:1398] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 19:57:37.038411 7084 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:37.038762 7085 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:37.038832 7085 ts_tablet_manager.cc:1367] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:57:37.039212 7084 tablet_bootstrap.cc:654] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.039207 7085 raft_consensus.cc:348] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.039325 7085 raft_consensus.cc:374] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.039363 7085 raft_consensus.cc:729] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.039485 7085 consensus_queue.cc:260] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.039757 7085 ts_tablet_manager.cc:1398] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:37.039918 7085 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:37.040573 7086 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:37.040663 7086 ts_tablet_manager.cc:1367] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.000s
I20240501 19:57:37.040964 7085 tablet_bootstrap.cc:654] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.041069 7086 raft_consensus.cc:348] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.041184 7086 raft_consensus.cc:374] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.041231 7086 raft_consensus.cc:729] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.041357 7086 consensus_queue.cc:260] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.041641 7086 ts_tablet_manager.cc:1398] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.041821 7086 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:37.042878 7085 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:37.042964 7085 ts_tablet_manager.cc:1367] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240501 19:57:37.043125 7086 tablet_bootstrap.cc:654] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.043361 7085 raft_consensus.cc:348] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.043464 7085 raft_consensus.cc:374] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.043509 7085 raft_consensus.cc:729] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.043601 7085 consensus_queue.cc:260] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.043870 7085 ts_tablet_manager.cc:1398] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.045197 6977 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:37.045271 6977 ts_tablet_manager.cc:1367] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.001s
I20240501 19:57:37.045642 6977 raft_consensus.cc:348] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.045769 6977 raft_consensus.cc:374] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.045805 6977 raft_consensus.cc:729] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.045922 6977 consensus_queue.cc:260] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.046196 6977 ts_tablet_manager.cc:1398] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240501 19:57:37.046219 7156 raft_consensus.cc:482] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:37.046350 6977 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:37.046335 7156 raft_consensus.cc:504] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.046854 7156 leader_election.cc:288] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:37.047010 6977 tablet_bootstrap.cc:654] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.047184 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b6181bae544db0b6bbdc6bcd01a40d" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:37.047282 5828 raft_consensus.cc:2454] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:37.047364 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b6181bae544db0b6bbdc6bcd01a40d" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:37.047458 5430 raft_consensus.cc:2454] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:37.048444 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b6181bae544db0b6bbdc6bcd01a40d" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:37.048477 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b6181bae544db0b6bbdc6bcd01a40d" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:37.048553 5960 raft_consensus.cc:2454] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:37.048916 5694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b6181bae544db0b6bbdc6bcd01a40d" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
W20240501 19:57:37.049221 5499 leader_election.cc:341] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:57:37.049290 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b6181bae544db0b6bbdc6bcd01a40d" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:37.049378 5296 raft_consensus.cc:2454] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
W20240501 19:57:37.049399 5499 leader_election.cc:341] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:57:37.049511 5499 leader_election.cc:302] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 9544b0e566c54a62a6135f3bff47c4f2, 9782508a47ba4403a39ca49fff7677fa, f5287d5ae4d84bd692b0f7985cd94171; no voters: 4d4855c096ac4e1e936abf15834332ae, a881063bd86945a8871c7c102d8235c8
I20240501 19:57:37.049666 7156 raft_consensus.cc:2781] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:37.049721 7156 raft_consensus.cc:482] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:37.049772 7156 raft_consensus.cc:3037] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.050410 6977 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:37.050506 6977 ts_tablet_manager.cc:1367] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240501 19:57:37.050654 7156 raft_consensus.cc:504] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.050858 6977 raft_consensus.cc:348] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.050946 6977 raft_consensus.cc:374] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.050967 6977 raft_consensus.cc:729] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.051085 6977 consensus_queue.cc:260] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.051229 7156 leader_election.cc:288] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Requested vote from peers 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:37.051370 6977 ts_tablet_manager.cc:1398] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.051509 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b6181bae544db0b6bbdc6bcd01a40d" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:37.051594 5430 raft_consensus.cc:3037] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.051642 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b6181bae544db0b6bbdc6bcd01a40d" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:37.051729 5960 raft_consensus.cc:3037] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.051724 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b6181bae544db0b6bbdc6bcd01a40d" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
W20240501 19:57:37.052031 5499 leader_election.cc:341] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:57:37.052390 5430 raft_consensus.cc:2454] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:37.052521 5960 raft_consensus.cc:2454] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:37.052646 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b6181bae544db0b6bbdc6bcd01a40d" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:37.052714 5694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b6181bae544db0b6bbdc6bcd01a40d" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:37.052742 5828 raft_consensus.cc:3037] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.053545 7086 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:37.053570 6990 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:37.053629 7086 ts_tablet_manager.cc:1367] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240501 19:57:37.053637 6990 ts_tablet_manager.cc:1367] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.000s
I20240501 19:57:37.053648 5828 raft_consensus.cc:2454] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
W20240501 19:57:37.053840 5499 leader_election.cc:341] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519): Illegal state: must be running to vote when last-logged opid is not known
I20240501 19:57:37.053997 7086 raft_consensus.cc:348] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.054023 6990 raft_consensus.cc:348] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.054091 7086 raft_consensus.cc:374] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.054118 6990 raft_consensus.cc:374] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.054121 7086 raft_consensus.cc:729] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.054150 6990 raft_consensus.cc:729] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.054222 7086 consensus_queue.cc:260] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.054296 6990 consensus_queue.cc:260] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.054481 5496 leader_election.cc:302] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 9544b0e566c54a62a6135f3bff47c4f2, 9782508a47ba4403a39ca49fff7677fa, f5287d5ae4d84bd692b0f7985cd94171; no voters: 4d4855c096ac4e1e936abf15834332ae, a881063bd86945a8871c7c102d8235c8
I20240501 19:57:37.054524 7086 ts_tablet_manager.cc:1398] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:37.054575 6990 ts_tablet_manager.cc:1398] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.054638 7148 raft_consensus.cc:2781] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:37.054670 7086 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:37.054705 7148 raft_consensus.cc:686] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Becoming Leader. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Running, Role: LEADER
I20240501 19:57:37.054819 7148 consensus_queue.cc:237] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.055356 7086 tablet_bootstrap.cc:654] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.055359 6990 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:37.056044 6990 tablet_bootstrap.cc:654] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.056509 7084 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:37.056581 7084 ts_tablet_manager.cc:1367] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240501 19:57:37.056632 7086 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:37.056727 7086 ts_tablet_manager.cc:1367] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:37.056933 7084 raft_consensus.cc:348] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.057032 7084 raft_consensus.cc:374] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.057060 7084 raft_consensus.cc:729] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.057085 7086 raft_consensus.cc:348] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.057183 7086 raft_consensus.cc:374] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.057147 7084 consensus_queue.cc:260] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.057214 7086 raft_consensus.cc:729] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.057309 7086 consensus_queue.cc:260] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.057428 7084 ts_tablet_manager.cc:1398] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.057547 7086 ts_tablet_manager.cc:1398] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.057564 7084 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:37.057868 5154 catalog_manager.cc:5484] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:37.058337 7084 tablet_bootstrap.cc:654] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.058202 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6b6181bae544db0b6bbdc6bcd01a40d" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:37.058676 5296 raft_consensus.cc:3037] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.059509 5296 raft_consensus.cc:2454] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:37.061055 6989 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:37.061139 6989 ts_tablet_manager.cc:1367] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.001s
I20240501 19:57:37.061193 7084 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:37.061295 7084 ts_tablet_manager.cc:1367] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240501 19:57:37.061560 6989 raft_consensus.cc:348] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.061669 6989 raft_consensus.cc:374] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.061698 6989 raft_consensus.cc:729] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.061645 7084 raft_consensus.cc:348] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.061739 7084 raft_consensus.cc:374] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.061798 7084 raft_consensus.cc:729] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.061808 6989 consensus_queue.cc:260] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.061884 7084 consensus_queue.cc:260] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.062073 6989 ts_tablet_manager.cc:1398] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:37.062157 7084 ts_tablet_manager.cc:1398] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:37.062804 6990 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:37.062861 6990 ts_tablet_manager.cc:1367] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240501 19:57:37.063140 6990 raft_consensus.cc:348] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.063208 6990 raft_consensus.cc:374] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.063226 6990 raft_consensus.cc:729] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.063331 6990 consensus_queue.cc:260] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.063589 6990 ts_tablet_manager.cc:1398] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.063717 7066 raft_consensus.cc:482] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:37.063798 7066 raft_consensus.cc:504] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.063861 6990 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:37.064344 7066 leader_election.cc:288] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:37.064739 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0df1e5e85294f60b8de22053dd577f7" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:37.064782 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0df1e5e85294f60b8de22053dd577f7" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:37.064844 5430 raft_consensus.cc:2454] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:37.064903 6990 tablet_bootstrap.cc:654] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.064898 5562 raft_consensus.cc:2454] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:37.065043 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0df1e5e85294f60b8de22053dd577f7" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:37.065135 5296 raft_consensus.cc:2454] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:37.065212 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0df1e5e85294f60b8de22053dd577f7" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:37.065315 5960 raft_consensus.cc:2454] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:37.065366 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0df1e5e85294f60b8de22053dd577f7" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:37.065446 6097 raft_consensus.cc:2454] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:37.065663 5631 leader_election.cc:302] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, 9782508a47ba4403a39ca49fff7677fa, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:37.066293 7066 raft_consensus.cc:2781] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:37.066344 7066 raft_consensus.cc:482] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:37.066375 7066 raft_consensus.cc:3037] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.067193 7066 raft_consensus.cc:504] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.067714 7066 leader_election.cc:288] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 election: Requested vote from peers 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:37.067840 5960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0df1e5e85294f60b8de22053dd577f7" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:37.067840 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0df1e5e85294f60b8de22053dd577f7" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:37.067914 6097 raft_consensus.cc:3037] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.067914 5960 raft_consensus.cc:3037] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.067981 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0df1e5e85294f60b8de22053dd577f7" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:37.067981 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0df1e5e85294f60b8de22053dd577f7" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:37.068064 5430 raft_consensus.cc:3037] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.068064 5296 raft_consensus.cc:3037] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.068614 5960 raft_consensus.cc:2454] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:37.068619 6097 raft_consensus.cc:2454] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:37.068765 5430 raft_consensus.cc:2454] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:37.068765 5296 raft_consensus.cc:2454] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:37.068991 5628 leader_election.cc:302] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, a881063bd86945a8871c7c102d8235c8, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:37.069026 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0df1e5e85294f60b8de22053dd577f7" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:37.069114 5562 raft_consensus.cc:3037] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.069307 5828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0df1e5e85294f60b8de22053dd577f7" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:37.069358 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0df1e5e85294f60b8de22053dd577f7" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:37.069402 5828 raft_consensus.cc:2454] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:37.069465 5827 raft_consensus.cc:3037] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.069757 7066 raft_consensus.cc:2781] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:37.069849 7066 raft_consensus.cc:686] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Becoming Leader. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Running, Role: LEADER
I20240501 19:57:37.069886 5562 raft_consensus.cc:2454] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:37.069970 7066 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.070317 5827 raft_consensus.cc:2454] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:37.071640 6990 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:37.071730 6990 ts_tablet_manager.cc:1367] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240501 19:57:37.071846 5147 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d4855c096ac4e1e936abf15834332ae (127.20.210.4). New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:37.072114 6990 raft_consensus.cc:348] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.072221 6990 raft_consensus.cc:374] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.072289 6990 raft_consensus.cc:729] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.072404 6990 consensus_queue.cc:260] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.072651 6990 ts_tablet_manager.cc:1398] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:37.081959 7169 raft_consensus.cc:482] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:37.082048 7169 raft_consensus.cc:504] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.082653 7169 leader_election.cc:288] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:37.082777 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1211024f45e142979fc1e5e8fff010ce" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:37.082873 5430 raft_consensus.cc:2454] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:37.082872 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1211024f45e142979fc1e5e8fff010ce" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:37.082952 5296 raft_consensus.cc:2454] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:37.082967 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1211024f45e142979fc1e5e8fff010ce" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:37.083037 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1211024f45e142979fc1e5e8fff010ce" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:37.083060 5827 raft_consensus.cc:2454] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:37.083117 5562 raft_consensus.cc:2454] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:37.083145 5694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1211024f45e142979fc1e5e8fff010ce" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:37.083245 5694 raft_consensus.cc:2454] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:37.083222 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1211024f45e142979fc1e5e8fff010ce" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:37.083324 6097 raft_consensus.cc:2454] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:37.083345 5898 leader_election.cc:302] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 9544b0e566c54a62a6135f3bff47c4f2, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:37.083487 7169 raft_consensus.cc:2781] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:37.083547 7169 raft_consensus.cc:482] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:37.083608 7169 raft_consensus.cc:3037] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.084276 7169 raft_consensus.cc:504] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.084816 7169 leader_election.cc:288] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [CANDIDATE]: Term 1 election: Requested vote from peers 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:37.084854 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1211024f45e142979fc1e5e8fff010ce" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:37.084975 5430 raft_consensus.cc:3037] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.085036 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1211024f45e142979fc1e5e8fff010ce" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:37.085119 5296 raft_consensus.cc:3037] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.085156 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1211024f45e142979fc1e5e8fff010ce" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:37.085186 5694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1211024f45e142979fc1e5e8fff010ce" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:37.085228 5827 raft_consensus.cc:3037] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.085258 5694 raft_consensus.cc:3037] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.085713 5430 raft_consensus.cc:2454] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:37.085819 5296 raft_consensus.cc:2454] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:37.085951 5827 raft_consensus.cc:2454] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:37.086076 5694 raft_consensus.cc:2454] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:37.086156 5896 leader_election.cc:302] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 9544b0e566c54a62a6135f3bff47c4f2, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:37.086349 7169 raft_consensus.cc:2781] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:37.086387 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1211024f45e142979fc1e5e8fff010ce" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:37.086424 7169 raft_consensus.cc:686] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Becoming Leader. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Running, Role: LEADER
I20240501 19:57:37.086470 6097 raft_consensus.cc:3037] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.086542 7169 consensus_queue.cc:237] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.087217 6097 raft_consensus.cc:2454] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:37.087415 5562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1211024f45e142979fc1e5e8fff010ce" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:37.087508 5562 raft_consensus.cc:3037] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.088078 5562 raft_consensus.cc:2454] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:37.088598 5152 catalog_manager.cc:5484] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6). New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } }
W20240501 19:57:37.193990 7203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:37.194209 7203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:37.197198 7203 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240501 19:57:37.263978 5827 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.264175 5296 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.264676 7170 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.265010 7170 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.265375 5694 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.265470 5430 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.265772 6097 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.265869 7170 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.266263 5562 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.266624 6910 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.266891 6910 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.267146 6910 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.270418 5828 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.272118 6097 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.272825 5960 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.273113 7072 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.273357 7072 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.279836 5562 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.280323 7066 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.286545 5430 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.287140 5296 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.287279 7066 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.293229 7067 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.293510 7067 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.298856 5962 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.299206 7156 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.304836 6095 raft_consensus.cc:3037] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.305930 6095 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.307121 7156 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.314592 5696 raft_consensus.cc:3037] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.317574 5696 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.317910 7148 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.326615 5429 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.327417 5828 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.328047 7153 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.329062 5297 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.329351 7148 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.329840 7156 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 162.898 ms
time per row: 0.0814492 ms
I20240501 19:57:37.414958 5152 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47970:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240501 19:57:37.415395 5152 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_6 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240501 19:57:37.420334 5276 tablet_service.cc:1467] Processing CreateTablet for tablet 2f89a8073ad64580b7aa8612204b3d38 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.420585 5276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f89a8073ad64580b7aa8612204b3d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.422459 7085 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:37.422770 5941 tablet_service.cc:1467] Processing CreateTablet for tablet cb5540ce8d3343a68a8bf7d8fff00589 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.423022 5941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5540ce8d3343a68a8bf7d8fff00589. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.423255 7085 tablet_bootstrap.cc:654] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.424206 5942 tablet_service.cc:1467] Processing CreateTablet for tablet c6ba23a7b4d24e0aa077e47798e890b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.424479 5942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ba23a7b4d24e0aa077e47798e890b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.426584 6987 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:37.427634 6987 tablet_bootstrap.cc:654] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.422770 5940 tablet_service.cc:1467] Processing CreateTablet for tablet 2f89a8073ad64580b7aa8612204b3d38 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.428398 5940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f89a8073ad64580b7aa8612204b3d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.430012 6077 tablet_service.cc:1467] Processing CreateTablet for tablet 2f89a8073ad64580b7aa8612204b3d38 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.430539 6075 tablet_service.cc:1467] Processing CreateTablet for tablet cb5540ce8d3343a68a8bf7d8fff00589 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.430939 6077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f89a8073ad64580b7aa8612204b3d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.431296 6075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5540ce8d3343a68a8bf7d8fff00589. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.432490 5541 tablet_service.cc:1467] Processing CreateTablet for tablet 2f89a8073ad64580b7aa8612204b3d38 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.432710 5541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f89a8073ad64580b7aa8612204b3d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.432772 5807 tablet_service.cc:1467] Processing CreateTablet for tablet cb5540ce8d3343a68a8bf7d8fff00589 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.433030 5807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5540ce8d3343a68a8bf7d8fff00589. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.433199 5808 tablet_service.cc:1467] Processing CreateTablet for tablet 2f89a8073ad64580b7aa8612204b3d38 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.433380 5808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f89a8073ad64580b7aa8612204b3d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.434631 5542 tablet_service.cc:1467] Processing CreateTablet for tablet c6ba23a7b4d24e0aa077e47798e890b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.434767 7084 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:37.434856 5542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ba23a7b4d24e0aa077e47798e890b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.435093 6990 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:37.435184 5809 tablet_service.cc:1467] Processing CreateTablet for tablet c6ba23a7b4d24e0aa077e47798e890b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.435395 5809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ba23a7b4d24e0aa077e47798e890b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.435613 7084 tablet_bootstrap.cc:654] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.435917 6990 tablet_bootstrap.cc:654] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.436717 5543 tablet_service.cc:1467] Processing CreateTablet for tablet cb5540ce8d3343a68a8bf7d8fff00589 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.430951 6076 tablet_service.cc:1467] Processing CreateTablet for tablet c6ba23a7b4d24e0aa077e47798e890b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.436918 5543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5540ce8d3343a68a8bf7d8fff00589. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.436971 6076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ba23a7b4d24e0aa077e47798e890b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.436991 6990 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:37.437055 6990 ts_tablet_manager.cc:1367] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:37.437191 6977 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:37.437431 6990 raft_consensus.cc:348] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.437536 6990 raft_consensus.cc:374] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.437570 6990 raft_consensus.cc:729] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.437697 6990 consensus_queue.cc:260] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.437885 6977 tablet_bootstrap.cc:654] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.437985 6990 ts_tablet_manager.cc:1398] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.438122 6990 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:37.438877 5410 tablet_service.cc:1467] Processing CreateTablet for tablet 2f89a8073ad64580b7aa8612204b3d38 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.438910 5277 tablet_service.cc:1467] Processing CreateTablet for tablet c6ba23a7b4d24e0aa077e47798e890b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.439067 6990 tablet_bootstrap.cc:654] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.439150 5410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f89a8073ad64580b7aa8612204b3d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.439208 5277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ba23a7b4d24e0aa077e47798e890b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.440025 6990 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:37.440090 6990 ts_tablet_manager.cc:1367] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240501 19:57:37.440066 5409 tablet_service.cc:1467] Processing CreateTablet for tablet c6ba23a7b4d24e0aa077e47798e890b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.440315 5409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ba23a7b4d24e0aa077e47798e890b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.440538 6990 raft_consensus.cc:348] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.440634 6990 raft_consensus.cc:374] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.440662 6990 raft_consensus.cc:729] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.440752 6990 consensus_queue.cc:260] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.441033 6990 ts_tablet_manager.cc:1398] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:37.441229 7086 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:37.441996 7086 tablet_bootstrap.cc:654] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.442164 5408 tablet_service.cc:1467] Processing CreateTablet for tablet cb5540ce8d3343a68a8bf7d8fff00589 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.442397 5408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5540ce8d3343a68a8bf7d8fff00589. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.443104 5675 tablet_service.cc:1467] Processing CreateTablet for tablet 2f89a8073ad64580b7aa8612204b3d38 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.443161 5674 tablet_service.cc:1467] Processing CreateTablet for tablet cb5540ce8d3343a68a8bf7d8fff00589 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.443358 5675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f89a8073ad64580b7aa8612204b3d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.443552 5673 tablet_service.cc:1467] Processing CreateTablet for tablet c6ba23a7b4d24e0aa077e47798e890b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.443760 5673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ba23a7b4d24e0aa077e47798e890b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.438928 7085 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:37.443954 7085 ts_tablet_manager.cc:1367] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240501 19:57:37.444353 7085 raft_consensus.cc:348] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.444455 7085 raft_consensus.cc:374] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.444486 7085 raft_consensus.cc:729] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.444609 7085 consensus_queue.cc:260] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.444896 7085 ts_tablet_manager.cc:1398] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.445060 7085 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:37.445924 6990 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8: Bootstrap starting.
I20240501 19:57:37.445938 6989 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:37.445996 7085 tablet_bootstrap.cc:654] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.446345 5674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5540ce8d3343a68a8bf7d8fff00589. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.446821 6989 tablet_bootstrap.cc:654] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.446949 6990 tablet_bootstrap.cc:654] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.447731 5276 tablet_service.cc:1467] Processing CreateTablet for tablet cb5540ce8d3343a68a8bf7d8fff00589 (DEFAULT_TABLE table=rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240501 19:57:37.447947 5276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5540ce8d3343a68a8bf7d8fff00589. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:37.449462 7086 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:37.449541 7086 ts_tablet_manager.cc:1367] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240501 19:57:37.449949 7086 raft_consensus.cc:348] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.450088 7086 raft_consensus.cc:374] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.450143 7086 raft_consensus.cc:729] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.450290 7086 consensus_queue.cc:260] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.450610 7086 ts_tablet_manager.cc:1398] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.450798 7086 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:37.450990 6977 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:37.451072 6977 ts_tablet_manager.cc:1367] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240501 19:57:37.451637 7086 tablet_bootstrap.cc:654] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.451547 6977 raft_consensus.cc:348] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.451678 6977 raft_consensus.cc:374] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.451709 6977 raft_consensus.cc:729] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.451836 6977 consensus_queue.cc:260] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.452134 6977 ts_tablet_manager.cc:1398] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.452314 6977 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:37.453099 6977 tablet_bootstrap.cc:654] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.454449 7086 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:37.454515 7086 ts_tablet_manager.cc:1367] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240501 19:57:37.454592 6990 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8: No bootstrap required, opened a new log
I20240501 19:57:37.454656 6990 ts_tablet_manager.cc:1367] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240501 19:57:37.454811 7084 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:37.454896 7084 ts_tablet_manager.cc:1367] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.001s
I20240501 19:57:37.454901 7086 raft_consensus.cc:348] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.454998 7086 raft_consensus.cc:374] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.455031 7086 raft_consensus.cc:729] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.455029 6990 raft_consensus.cc:348] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.455119 6990 raft_consensus.cc:374] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.455150 6990 raft_consensus.cc:729] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a881063bd86945a8871c7c102d8235c8, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.455132 7086 consensus_queue.cc:260] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.455256 6990 consensus_queue.cc:260] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.455368 7086 ts_tablet_manager.cc:1398] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.455497 7086 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:37.455488 6990 ts_tablet_manager.cc:1398] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:37.455440 7084 raft_consensus.cc:348] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.455585 7084 raft_consensus.cc:374] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.455622 7084 raft_consensus.cc:729] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.455736 7084 consensus_queue.cc:260] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.456065 7084 ts_tablet_manager.cc:1398] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:37.456246 7084 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:37.456369 7086 tablet_bootstrap.cc:654] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.457084 6987 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:37.457156 6987 ts_tablet_manager.cc:1367] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20240501 19:57:37.457289 7084 tablet_bootstrap.cc:654] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.457504 6987 raft_consensus.cc:348] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.457602 6987 raft_consensus.cc:374] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.457625 6987 raft_consensus.cc:729] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.457737 6987 consensus_queue.cc:260] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.457978 6987 ts_tablet_manager.cc:1398] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.458119 6987 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:37.465132 6987 tablet_bootstrap.cc:654] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.466595 7086 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa: No bootstrap required, opened a new log
I20240501 19:57:37.466665 7086 ts_tablet_manager.cc:1367] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240501 19:57:37.466997 7086 raft_consensus.cc:348] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.467118 7086 raft_consensus.cc:374] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.467144 7086 raft_consensus.cc:729] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.467245 7086 consensus_queue.cc:260] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.467559 7086 ts_tablet_manager.cc:1398] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.467594 6977 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:37.467664 6977 ts_tablet_manager.cc:1367] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240501 19:57:37.468041 6977 raft_consensus.cc:348] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.468149 6977 raft_consensus.cc:374] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.468187 6977 raft_consensus.cc:729] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.468333 6977 consensus_queue.cc:260] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.468602 6977 ts_tablet_manager.cc:1398] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.468768 6977 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:37.469892 6977 tablet_bootstrap.cc:654] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.471462 6989 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:37.471565 6989 ts_tablet_manager.cc:1367] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.002s
I20240501 19:57:37.471700 6987 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:37.471776 6987 ts_tablet_manager.cc:1367] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20240501 19:57:37.472059 6989 raft_consensus.cc:348] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.472139 6987 raft_consensus.cc:348] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.472208 6989 raft_consensus.cc:374] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.472242 6987 raft_consensus.cc:374] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.472254 6989 raft_consensus.cc:729] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.472277 6987 raft_consensus.cc:729] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.472379 6987 consensus_queue.cc:260] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.472385 6989 consensus_queue.cc:260] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.472607 6987 ts_tablet_manager.cc:1398] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.472738 6989 ts_tablet_manager.cc:1398] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.472743 6987 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:37.472918 6989 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:37.473673 6987 tablet_bootstrap.cc:654] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.474025 6989 tablet_bootstrap.cc:654] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.479373 7084 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:37.479465 7084 ts_tablet_manager.cc:1367] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.002s
I20240501 19:57:37.479895 6987 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2: No bootstrap required, opened a new log
I20240501 19:57:37.479954 6987 ts_tablet_manager.cc:1367] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240501 19:57:37.479939 7084 raft_consensus.cc:348] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.480073 7084 raft_consensus.cc:374] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.480109 7084 raft_consensus.cc:729] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.480291 7084 consensus_queue.cc:260] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.480319 6987 raft_consensus.cc:348] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.480433 6987 raft_consensus.cc:374] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.480458 6987 raft_consensus.cc:729] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.480553 6987 consensus_queue.cc:260] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.480660 7084 ts_tablet_manager.cc:1398] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:37.480818 6987 ts_tablet_manager.cc:1398] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.480856 7084 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171: Bootstrap starting.
I20240501 19:57:37.483141 6977 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3: No bootstrap required, opened a new log
I20240501 19:57:37.483217 6977 ts_tablet_manager.cc:1367] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240501 19:57:37.483554 6977 raft_consensus.cc:348] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.483644 6977 raft_consensus.cc:374] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.483666 6977 raft_consensus.cc:729] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.483817 6977 consensus_queue.cc:260] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.484053 6977 ts_tablet_manager.cc:1398] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.484575 7084 tablet_bootstrap.cc:654] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.488286 7084 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171: No bootstrap required, opened a new log
I20240501 19:57:37.488444 7084 ts_tablet_manager.cc:1367] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.002s
I20240501 19:57:37.488956 7084 raft_consensus.cc:348] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.489107 7084 raft_consensus.cc:374] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.489153 7084 raft_consensus.cc:729] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.489303 7084 consensus_queue.cc:260] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.489768 7084 ts_tablet_manager.cc:1398] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:37.490489 7085 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:37.490579 7085 ts_tablet_manager.cc:1367] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.046s user 0.001s sys 0.000s
I20240501 19:57:37.490605 6989 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:37.490676 6989 ts_tablet_manager.cc:1367] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.002s
I20240501 19:57:37.491026 6989 raft_consensus.cc:348] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.491026 7085 raft_consensus.cc:348] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.491145 6989 raft_consensus.cc:374] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.491159 7085 raft_consensus.cc:374] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.491182 6989 raft_consensus.cc:729] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.491194 7085 raft_consensus.cc:729] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.491307 6989 consensus_queue.cc:260] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.491307 7085 consensus_queue.cc:260] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.491573 7085 ts_tablet_manager.cc:1398] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.491577 6989 ts_tablet_manager.cc:1398] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240501 19:57:37.491724 6989 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:37.491750 7085 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:37.492553 7085 tablet_bootstrap.cc:654] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.492553 6989 tablet_bootstrap.cc:654] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae: Neither blocks nor log segments found. Creating new log.
I20240501 19:57:37.494748 6989 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae: No bootstrap required, opened a new log
I20240501 19:57:37.494835 6989 ts_tablet_manager.cc:1367] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240501 19:57:37.495363 6989 raft_consensus.cc:348] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.495497 6989 raft_consensus.cc:374] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.495543 6989 raft_consensus.cc:729] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.495676 6989 consensus_queue.cc:260] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.496033 6989 ts_tablet_manager.cc:1398] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.498940 7085 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0: No bootstrap required, opened a new log
I20240501 19:57:37.499009 7085 ts_tablet_manager.cc:1367] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240501 19:57:37.499370 7085 raft_consensus.cc:348] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.499467 7085 raft_consensus.cc:374] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240501 19:57:37.499492 7085 raft_consensus.cc:729] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: FOLLOWER
I20240501 19:57:37.499599 7085 consensus_queue.cc:260] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.499878 7085 ts_tablet_manager.cc:1398] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:37.517289 7170 raft_consensus.cc:482] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:37.517446 7170 raft_consensus.cc:504] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.518390 7170 leader_election.cc:288] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:37.518435 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f89a8073ad64580b7aa8612204b3d38" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:37.518570 5296 raft_consensus.cc:2454] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:37.519033 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f89a8073ad64580b7aa8612204b3d38" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:37.519194 6096 raft_consensus.cc:2454] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:37.519162 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f89a8073ad64580b7aa8612204b3d38" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:37.519299 5827 raft_consensus.cc:2454] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:37.519400 5694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f89a8073ad64580b7aa8612204b3d38" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:37.519508 5694 raft_consensus.cc:2454] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:37.519513 5429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f89a8073ad64580b7aa8612204b3d38" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:37.519635 5429 raft_consensus.cc:2454] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:37.519773 5898 leader_election.cc:302] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, a881063bd86945a8871c7c102d8235c8, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:37.520010 7253 raft_consensus.cc:482] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:37.520092 7253 raft_consensus.cc:504] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.520315 5563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f89a8073ad64580b7aa8612204b3d38" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:37.520473 5563 raft_consensus.cc:2454] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 0.
I20240501 19:57:37.520699 7253 leader_election.cc:288] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:37.521176 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5540ce8d3343a68a8bf7d8fff00589" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:37.521190 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5540ce8d3343a68a8bf7d8fff00589" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:37.521284 5827 raft_consensus.cc:2454] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:37.521296 6096 raft_consensus.cc:2454] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:37.521651 5429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5540ce8d3343a68a8bf7d8fff00589" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:37.521662 5563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5540ce8d3343a68a8bf7d8fff00589" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171" is_pre_election: true
I20240501 19:57:37.521755 5429 raft_consensus.cc:2454] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:37.521785 5563 raft_consensus.cc:2454] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:37.522012 5628 leader_election.cc:302] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9782508a47ba4403a39ca49fff7677fa, a881063bd86945a8871c7c102d8235c8; no voters:
I20240501 19:57:37.522186 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5540ce8d3343a68a8bf7d8fff00589" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:37.522326 5296 raft_consensus.cc:2454] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:37.522471 7253 raft_consensus.cc:2781] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:37.522519 7253 raft_consensus.cc:482] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:37.522564 7253 raft_consensus.cc:3037] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.523258 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5540ce8d3343a68a8bf7d8fff00589" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:37.523360 5962 raft_consensus.cc:2454] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 0.
I20240501 19:57:37.523488 7253 raft_consensus.cc:504] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.523867 7170 raft_consensus.cc:2781] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:37.523938 7170 raft_consensus.cc:482] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:37.523999 7170 raft_consensus.cc:3037] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.524137 7253 leader_election.cc:288] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 election: Requested vote from peers 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:37.524525 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5540ce8d3343a68a8bf7d8fff00589" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:37.524626 5827 raft_consensus.cc:3037] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.524768 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5540ce8d3343a68a8bf7d8fff00589" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:37.524868 6096 raft_consensus.cc:3037] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.525215 7170 raft_consensus.cc:504] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.525578 5827 raft_consensus.cc:2454] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:37.525790 6096 raft_consensus.cc:2454] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:37.525966 7170 leader_election.cc:288] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [CANDIDATE]: Term 1 election: Requested vote from peers ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815)
I20240501 19:57:37.526057 5563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5540ce8d3343a68a8bf7d8fff00589" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:37.526142 5563 raft_consensus.cc:3037] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.526625 5297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f89a8073ad64580b7aa8612204b3d38" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:37.526715 5297 raft_consensus.cc:3037] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.526728 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f89a8073ad64580b7aa8612204b3d38" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:37.526829 5827 raft_consensus.cc:3037] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.527081 5563 raft_consensus.cc:2454] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:37.527722 5295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5540ce8d3343a68a8bf7d8fff00589" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:37.527777 5297 raft_consensus.cc:2454] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:37.527814 5295 raft_consensus.cc:3037] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.527860 5827 raft_consensus.cc:2454] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:37.528458 5429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f89a8073ad64580b7aa8612204b3d38" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:37.528488 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5540ce8d3343a68a8bf7d8fff00589" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:37.528564 5429 raft_consensus.cc:3037] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.528594 5430 raft_consensus.cc:3037] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.528684 5295 raft_consensus.cc:2454] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:37.529500 5430 raft_consensus.cc:2454] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:37.529593 5429 raft_consensus.cc:2454] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:37.529734 5631 leader_election.cc:302] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, ad52038f1d38489ea464a8fbc72daac0, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:37.529888 5895 leader_election.cc:302] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 9544b0e566c54a62a6135f3bff47c4f2, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:37.530066 7170 raft_consensus.cc:2781] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:37.530308 7170 raft_consensus.cc:686] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Becoming Leader. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Running, Role: LEADER
I20240501 19:57:37.530498 7170 consensus_queue.cc:237] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.530861 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f89a8073ad64580b7aa8612204b3d38" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:37.530972 6097 raft_consensus.cc:3037] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.531893 6097 raft_consensus.cc:2454] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:37.532117 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5540ce8d3343a68a8bf7d8fff00589" candidate_uuid: "4d4855c096ac4e1e936abf15834332ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:37.532227 5962 raft_consensus.cc:3037] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.533077 5962 raft_consensus.cc:2454] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d4855c096ac4e1e936abf15834332ae in term 1.
I20240501 19:57:37.533771 5694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f89a8073ad64580b7aa8612204b3d38" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:37.533864 5694 raft_consensus.cc:3037] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.534395 5154 catalog_manager.cc:5484] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6). New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:37.534693 5694 raft_consensus.cc:2454] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:37.535423 5563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f89a8073ad64580b7aa8612204b3d38" candidate_uuid: "9544b0e566c54a62a6135f3bff47c4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:37.535518 5563 raft_consensus.cc:3037] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.535634 7067 raft_consensus.cc:2781] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:37.535718 7067 raft_consensus.cc:686] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Becoming Leader. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Running, Role: LEADER
I20240501 19:57:37.535898 7067 consensus_queue.cc:237] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:37.536312 5563 raft_consensus.cc:2454] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9544b0e566c54a62a6135f3bff47c4f2 in term 1.
I20240501 19:57:37.538062 5152 catalog_manager.cc:5484] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d4855c096ac4e1e936abf15834332ae (127.20.210.4). New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:37.567739 7150 raft_consensus.cc:482] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240501 19:57:37.567844 7150 raft_consensus.cc:504] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.568542 7150 leader_election.cc:288] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
I20240501 19:57:37.568619 5694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ba23a7b4d24e0aa077e47798e890b8" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae" is_pre_election: true
I20240501 19:57:37.568746 5694 raft_consensus.cc:2454] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:37.569048 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ba23a7b4d24e0aa077e47798e890b8" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3" is_pre_election: true
I20240501 19:57:37.569083 5429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ba23a7b4d24e0aa077e47798e890b8" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa" is_pre_election: true
I20240501 19:57:37.569167 5827 raft_consensus.cc:2454] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:37.569185 5429 raft_consensus.cc:2454] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:37.569557 5497 leader_election.cc:302] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9782508a47ba4403a39ca49fff7677fa, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:37.569842 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ba23a7b4d24e0aa077e47798e890b8" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8" is_pre_election: true
I20240501 19:57:37.569903 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ba23a7b4d24e0aa077e47798e890b8" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0" is_pre_election: true
I20240501 19:57:37.569958 6096 raft_consensus.cc:2454] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:37.570017 5296 raft_consensus.cc:2454] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:37.570601 7150 raft_consensus.cc:2781] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240501 19:57:37.570659 7150 raft_consensus.cc:482] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240501 19:57:37.570698 7150 raft_consensus.cc:3037] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.571568 7150 raft_consensus.cc:504] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.572124 5827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ba23a7b4d24e0aa077e47798e890b8" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1865147cd5a346ad8549f18fa8cc26c3"
I20240501 19:57:37.572124 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ba23a7b4d24e0aa077e47798e890b8" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2" is_pre_election: true
I20240501 19:57:37.572237 5962 raft_consensus.cc:2454] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 0.
I20240501 19:57:37.572227 5827 raft_consensus.cc:3037] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.572197 5694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ba23a7b4d24e0aa077e47798e890b8" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:37.572325 5694 raft_consensus.cc:3037] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.572959 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ba23a7b4d24e0aa077e47798e890b8" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:37.573068 5962 raft_consensus.cc:3037] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.573148 5827 raft_consensus.cc:2454] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:37.573424 5694 raft_consensus.cc:2454] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:37.573923 5962 raft_consensus.cc:2454] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:37.574184 5499 leader_election.cc:302] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1865147cd5a346ad8549f18fa8cc26c3, 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, f5287d5ae4d84bd692b0f7985cd94171; no voters:
I20240501 19:57:37.574602 7150 leader_election.cc:288] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [CANDIDATE]: Term 1 election: Requested vote from peers 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499), 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
I20240501 19:57:37.574693 5296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ba23a7b4d24e0aa077e47798e890b8" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:37.574785 5296 raft_consensus.cc:3037] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.575500 5296 raft_consensus.cc:2454] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:37.574620 7154 raft_consensus.cc:2781] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Leader election won for term 1
I20240501 19:57:37.575887 7154 raft_consensus.cc:686] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Becoming Leader. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Running, Role: LEADER
I20240501 19:57:37.576045 7154 consensus_queue.cc:237] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:37.577193 5429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ba23a7b4d24e0aa077e47798e890b8" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:37.577324 5429 raft_consensus.cc:3037] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.578294 5429 raft_consensus.cc:2454] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:37.578753 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ba23a7b4d24e0aa077e47798e890b8" candidate_uuid: "f5287d5ae4d84bd692b0f7985cd94171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:37.578856 6096 raft_consensus.cc:3037] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 0 FOLLOWER]: Advancing to term 1
I20240501 19:57:37.579787 6096 raft_consensus.cc:2454] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5287d5ae4d84bd692b0f7985cd94171 in term 1.
I20240501 19:57:37.579717 5152 catalog_manager.cc:5484] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3). New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } }
W20240501 19:57:37.694213 7279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:37.694401 7279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:37.697081 7279 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240501 19:57:37.777206 5296 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.777568 5429 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.778436 5828 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.778713 7170 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.779951 5827 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.779965 7253 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.780257 5429 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.780529 5296 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.781188 6096 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.781239 6097 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.781528 5563 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.781558 7170 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.782037 7067 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.782039 7170 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.782595 5694 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.783869 7253 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.785290 5962 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.786500 7170 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.787019 7072 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.787215 7072 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.787395 7072 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.787472 5563 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.787840 5429 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.787974 5696 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.788285 7150 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.788656 6097 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.791152 7169 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.791541 7154 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.791842 7154 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.793471 5960 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.796612 7169 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.810680 5827 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.812453 5296 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240501 19:57:37.813707 7154 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.814523 7150 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.820896 7154 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240501 19:57:37.897899 6929 mvcc.cc:205] Tried to move back new op lower bound from 7022974803064537088 to 7022974802050899968. Current Snapshot: MvccSnapshot[applied={T|T < 7022974803050037248}]
INSERT report
rows total: 2000
time total: 191.349 ms
time per row: 0.0956743 ms
I20240501 19:57:37.919076 7328 mvcc.cc:205] Tried to move back new op lower bound from 7022974803090022400 to 7022974802234695680. Current Snapshot: MvccSnapshot[applied={T|T < 7022974803090022400}]
I20240501 19:57:37.930985 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.8:41983
--local_ip_for_outbound_sockets=127.20.210.8
--tserver_master_addrs=127.20.210.62:45969
--webserver_port=35027
--webserver_interface=127.20.210.8
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240501 19:57:38.032953 7335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:38.033118 7335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:38.033200 7335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:38.036083 7335 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:38.036154 7335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:38.036213 7335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.8
I20240501 19:57:38.039669 7335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.8:41983
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.20.210.8
--webserver_port=35027
--tserver_master_addrs=127.20.210.62:45969
--never_fsync=true
--heap_profile_path=/tmp/kudu.7335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.8
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:38.040315 7335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:38.041253 7335 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:38.049401 7343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:38.049819 7340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:38.050549 7341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:38.051367 7335 server_base.cc:1034] running on GCE node
I20240501 19:57:38.051749 7335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:38.052289 7335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:38.053465 7335 hybrid_clock.cc:648] HybridClock initialized: now 1714593458053452 us; error 32 us; skew 500 ppm
I20240501 19:57:38.055088 7335 webserver.cc:460] Webserver started at http://127.20.210.8:35027/ using document root <none> and password file <none>
I20240501 19:57:38.055542 7335 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:38.055616 7335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:38.058342 7335 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20240501 19:57:38.059978 7350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:38.060869 7335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:38.060966 7335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "363f4ac4964f41a6ba0f98d1f487db83"
format_stamp: "Formatted at 2024-05-01 19:57:29 on dist-test-slave-vgh4"
I20240501 19:57:38.061304 7335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:38.074682 7335 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:38.075119 7335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:38.075246 7335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:38.075647 7335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:38.076337 7335 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:38.076383 7335 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:38.076433 7335 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:38.076458 7335 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:38.084389 7335 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.8:41983
I20240501 19:57:38.084420 7463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.8:41983 every 8 connection(s)
I20240501 19:57:38.085206 7335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240501 19:57:38.086016 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 7335
I20240501 19:57:38.086426 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.9:45861
--local_ip_for_outbound_sockets=127.20.210.9
--tserver_master_addrs=127.20.210.62:45969
--webserver_port=37435
--webserver_interface=127.20.210.9
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:38.093947 7464 heartbeater.cc:347] Connected to a master server at 127.20.210.62:45969
I20240501 19:57:38.094144 7464 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:38.094606 7464 heartbeater.cc:511] Master 127.20.210.62:45969 requested a full tablet report, sending...
I20240501 19:57:38.095711 5154 ts_manager.cc:194] Re-registered known tserver with Master: 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983)
I20240501 19:57:38.096377 5154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.8:51829
W20240501 19:57:38.201251 7467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:38.201447 7467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:38.201493 7467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:38.204653 7467 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:38.204710 7467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:38.204810 7467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.9
I20240501 19:57:38.208489 7467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.9:45861
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.20.210.9
--webserver_port=37435
--tserver_master_addrs=127.20.210.62:45969
--never_fsync=true
--heap_profile_path=/tmp/kudu.7467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.9
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:38.209322 7467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:38.210469 7467 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:38.218384 7475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:38.219022 7472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:38.219583 7473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:38.220410 7467 server_base.cc:1034] running on GCE node
I20240501 19:57:38.220723 7467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:38.221252 7467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:38.222415 7467 hybrid_clock.cc:648] HybridClock initialized: now 1714593458222397 us; error 28 us; skew 500 ppm
I20240501 19:57:38.224452 7467 webserver.cc:460] Webserver started at http://127.20.210.9:37435/ using document root <none> and password file <none>
I20240501 19:57:38.225056 7467 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:38.225144 7467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:38.228305 7467 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 19:57:38.230095 7481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:38.231103 7467 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:38.231204 7467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "143c54da1456472dadbf1d5d65e0a6f6"
format_stamp: "Formatted at 2024-05-01 19:57:30 on dist-test-slave-vgh4"
I20240501 19:57:38.231580 7467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:38.269874 7467 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:38.270467 7467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:38.270620 7467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:38.271119 7467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:38.271981 7467 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:38.272024 7467 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:38.272060 7467 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:38.272089 7467 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:38.280144 7467 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.9:45861
I20240501 19:57:38.280161 7595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.9:45861 every 8 connection(s)
I20240501 19:57:38.281225 7467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240501 19:57:38.283958 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 7467
I20240501 19:57:38.284318 21320 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
/tmp/dist-test-taskNtLy6c/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/logs
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.210.10:40603
--local_ip_for_outbound_sockets=127.20.210.10
--tserver_master_addrs=127.20.210.62:45969
--webserver_port=44709
--webserver_interface=127.20.210.10
--builtin_ntp_servers=127.20.210.20:33517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240501 19:57:38.291134 7596 heartbeater.cc:347] Connected to a master server at 127.20.210.62:45969
I20240501 19:57:38.291329 7596 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:38.291843 7596 heartbeater.cc:511] Master 127.20.210.62:45969 requested a full tablet report, sending...
I20240501 19:57:38.292780 5154 ts_manager.cc:194] Re-registered known tserver with Master: 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861)
I20240501 19:57:38.293278 5154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.9:50153
W20240501 19:57:38.395160 7599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240501 19:57:38.395311 7599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240501 19:57:38.395344 7599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240501 19:57:38.397964 7599 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240501 19:57:38.398012 7599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240501 19:57:38.398082 7599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.210.10
I20240501 19:57:38.401326 7599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.210.20:33517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data
--fs_wal_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.210.10:40603
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--webserver_interface=127.20.210.10
--webserver_port=44709
--tserver_master_addrs=127.20.210.62:45969
--never_fsync=true
--heap_profile_path=/tmp/kudu.7599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.210.10
--log_dir=/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 01 May 2024 19:43:13 UTC on 1a15035a6830
build id 77
I20240501 19:57:38.401999 7599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240501 19:57:38.402952 7599 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240501 19:57:38.410785 7606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:38.411324 7608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240501 19:57:38.411999 7605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240501 19:57:38.412711 7599 server_base.cc:1034] running on GCE node
I20240501 19:57:38.412974 7599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240501 19:57:38.413394 7599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240501 19:57:38.414520 7599 hybrid_clock.cc:648] HybridClock initialized: now 1714593458414507 us; error 25 us; skew 500 ppm
I20240501 19:57:38.416101 7599 webserver.cc:460] Webserver started at http://127.20.210.10:44709/ using document root <none> and password file <none>
I20240501 19:57:38.416631 7599 fs_manager.cc:359] Metadata directory not provided
I20240501 19:57:38.416724 7599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240501 19:57:38.419988 7599 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240501 19:57:38.421730 7615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:38.422719 7599 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:38.422829 7599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data,/tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal
uuid: "16737daa1d234f35826143a1aa29dc0b"
format_stamp: "Formatted at 2024-05-01 19:57:30 on dist-test-slave-vgh4"
I20240501 19:57:38.423235 7599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal
metadata directory: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal
1 data directories: /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240501 19:57:38.438571 7599 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240501 19:57:38.439025 7599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240501 19:57:38.439150 7599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240501 19:57:38.439563 7599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240501 19:57:38.440377 7599 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240501 19:57:38.440438 7599 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:38.440487 7599 ts_tablet_manager.cc:590] Registered 0 tablets
I20240501 19:57:38.440511 7599 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240501 19:57:38.447800 7599 rpc_server.cc:307] RPC server started. Bound to: 127.20.210.10:40603
I20240501 19:57:38.447813 7729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.210.10:40603 every 8 connection(s)
I20240501 19:57:38.448648 7599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/data/info.pb
I20240501 19:57:38.451530 21320 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNtLy6c/build/debug/bin/kudu as pid 7599
I20240501 19:57:38.457089 7730 heartbeater.cc:347] Connected to a master server at 127.20.210.62:45969
I20240501 19:57:38.457273 7730 heartbeater.cc:464] Registering TS with master...
I20240501 19:57:38.457679 7730 heartbeater.cc:511] Master 127.20.210.62:45969 requested a full tablet report, sending...
I20240501 19:57:38.458537 5154 ts_manager.cc:194] Re-registered known tserver with Master: 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603)
I20240501 19:57:38.459062 5154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.210.10:37715
I20240501 19:57:38.646716 7530 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:38.647243 7664 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:38.661270 5809 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:38.665218 5276 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:38.672763 5674 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:38.675522 7398 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:38.676282 6076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:38.681025 5408 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:38.691661 5942 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:38.718286 5543 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:38.814349 7530 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:38.817184 7664 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:38.822057 5809 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:38.824179 7398 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:38.828794 5674 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:38.832840 5942 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:38.856572 5543 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:38.858922 5276 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:38.859001 5408 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:38.872237 6076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:38.995908 7530 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:38.998700 7664 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:39.000604 5674 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:39.008824 7398 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:39.019846 5408 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:39.020207 5942 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:39.030207 5809 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:39.034722 6076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:39.050558 5543 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:39.053170 5276 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:39.091328 5828 consensus_queue.cc:237] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:39.093254 6097 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.093540 5429 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.093570 5694 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.093909 7609 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.094539 5962 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.094548 5296 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.095166 6972 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.095409 5563 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.095460 6972 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.095674 7609 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.095780 6972 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.096273 7609 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.098237 7464 heartbeater.cc:503] Master 127.20.210.62:45969 was elected leader, sending a full tablet report...
I20240501 19:57:39.098835 7883 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.099159 5962 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.100059 6097 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.101569 5563 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.101862 5429 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.104033 5150 catalog_manager.cc:5484] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.105180 5694 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.107388 5296 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.110332 5828 consensus_queue.cc:237] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:39.112746 5429 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.112800 5296 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.113360 5694 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.113571 5563 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.113543 7754 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.113648 6097 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.113857 5962 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.113857 7754 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.114089 6972 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.114475 7883 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.114681 7882 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.116530 7754 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.116894 5563 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.116976 7882 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.117518 5962 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.118404 5139 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet d8a84349ba37496289351ac689f77eab with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:39.119097 5429 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.119415 5296 consensus_queue.cc:237] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:39.119630 6097 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.120000 5153 catalog_manager.cc:5484] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:39.120412 5295 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.121570 5694 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.124008 5563 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.124413 7600 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.124670 5429 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.125571 5962 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.125742 5828 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.125845 7600 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.126154 6906 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.126838 6906 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.128422 6097 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.128495 5694 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.128815 7600 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.128974 6906 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240501 19:57:39.130167 5762 consensus_peers.cc:485] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 -> Peer 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): Couldn't send request to peer 363f4ac4964f41a6ba0f98d1f487db83. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8a84349ba37496289351ac689f77eab. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:39.130568 6906 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.132699 5960 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.133000 5429 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.133167 5147 catalog_manager.cc:5484] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:39.134578 5563 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.136418 6097 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.136562 5694 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.138775 5828 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.140537 5296 consensus_queue.cc:237] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.142217 5828 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.142673 7600 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.142768 6097 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.143455 5429 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.143455 5694 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.143800 7600 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.144125 7600 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.144446 5563 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.144979 7600 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.145268 7600 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.146283 7600 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.147738 5563 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.147855 5429 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.148262 5138 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 913563ed5f904cc28ff15148d9a5e5a2 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:39.148785 5962 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.148656 5828 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.150923 5152 catalog_manager.cc:5484] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.155128 7912 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.158176 5962 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
W20240501 19:57:39.160382 5231 consensus_peers.cc:485] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 913563ed5f904cc28ff15148d9a5e5a2. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:39.161027 6097 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.162220 5563 consensus_queue.cc:237] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:39.163127 5694 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.165868 5429 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.166260 7150 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.166704 6097 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.167770 7154 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.168242 5296 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.168663 5962 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.168757 5694 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.169242 7154 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.169255 5828 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.169483 7926 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.169690 7150 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.169792 7154 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.172011 5962 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.172250 7150 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.172730 5296 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.174922 5828 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.175400 6097 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.175364 5429 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.176738 5694 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.177848 5153 catalog_manager.cc:5484] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.184470 5563 consensus_queue.cc:237] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.186746 6095 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.187023 5694 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.187083 5430 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.187367 7154 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.187469 5960 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.187713 7154 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.188027 7154 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.188349 7154 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.188813 5296 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.188844 5828 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.189196 7925 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.189396 7926 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.190877 7150 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.192005 5960 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.193382 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet d3937a2d841440d793ba2637f1af060f with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:39.194090 5295 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.194196 6095 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.194864 5694 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.195611 5430 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.196910 5147 catalog_manager.cc:5484] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.198392 5828 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
W20240501 19:57:39.198693 5497 consensus_peers.cc:485] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3937a2d841440d793ba2637f1af060f. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:39.204145 5295 consensus_queue.cc:237] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } }
I20240501 19:57:39.205730 5828 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.205797 5430 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.205814 5694 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.206007 5563 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.206144 7913 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.206490 6095 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.206514 7915 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.206753 7912 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.206897 7915 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.207279 7912 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.208635 5960 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.208973 7912 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.209213 6906 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } }
I20240501 19:57:39.210212 5960 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } }
I20240501 19:57:39.210813 6095 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } }
I20240501 19:57:39.211336 5430 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } }
I20240501 19:57:39.212419 5694 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } }
I20240501 19:57:39.212689 5563 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } }
I20240501 19:57:39.213220 5153 catalog_manager.cc:5484] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } }
I20240501 19:57:39.217042 5828 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } }
I20240501 19:57:39.221321 5295 consensus_queue.cc:237] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:39.223137 5563 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.223137 5694 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.223218 5430 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.223496 7600 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.223604 5960 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.223786 7600 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.224376 7600 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.224496 6906 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.227535 7600 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
W20240501 19:57:39.228607 5231 consensus_peers.cc:485] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 95e8db3b5d81468098c01c32c4d22e80. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:39.228611 6095 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.228868 5138 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 95e8db3b5d81468098c01c32c4d22e80 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:39.229285 5828 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.229395 5429 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.229632 7912 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.229733 7915 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.229666 5960 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.230589 5694 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.231133 6095 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.231400 5563 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.232625 5828 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.233469 5147 catalog_manager.cc:5484] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.240599 5563 consensus_queue.cc:237] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:39.241999 6095 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.242105 5960 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.242285 5429 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.242693 5828 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.242733 7154 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.242733 5694 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.242906 7933 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.243065 7150 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.243192 7932 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.243320 7925 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.243448 5295 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.243839 7150 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.246070 7933 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.246487 5429 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.246845 5295 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.247881 5960 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.248881 5694 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.249816 5148 catalog_manager.cc:5484] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:39.250561 5828 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.254452 5563 consensus_queue.cc:237] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.256064 5295 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.256135 5429 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.256232 5828 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.256486 7932 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.256635 7926 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240501 19:57:39.256840 5497 consensus_peers.cc:485] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9ecbb3dec08047478ded9809f3a0b859. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:39.257015 7932 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.259003 6095 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.259043 5694 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.259402 7154 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.259503 7926 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.260105 7932 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.260326 5828 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.261648 6095 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.262850 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 9ecbb3dec08047478ded9809f3a0b859 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:39.263145 5962 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.263343 5960 consensus_queue.cc:237] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:39.263598 5295 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.263963 5694 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.264258 5147 catalog_manager.cc:5484] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.265027 5429 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.265483 7931 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.267557 6097 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.267576 5563 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.268503 5694 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.268895 6095 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.269834 7971 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.270048 7170 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.270398 7170 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.270615 5828 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.270929 5429 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.271081 7170 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.272277 7170 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.274130 5295 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.274524 7971 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.274824 5960 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.275643 7170 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.277105 5429 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.277240 5694 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.279289 5829 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.279546 5147 catalog_manager.cc:5484] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.282410 5563 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.282447 6095 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.290737 5960 consensus_queue.cc:237] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:39.293946 5829 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.294315 5694 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:39.294520 6095 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.294731 5696 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.295071 5563 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.295137 7170 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.295404 7170 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.295846 7596 heartbeater.cc:503] Master 127.20.210.62:45969 was elected leader, sending a full tablet report...
I20240501 19:57:39.296114 5429 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.297158 5563 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.297329 5296 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.297714 7170 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.297997 7170 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.298141 5429 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.298333 7253 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.298645 7253 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.298951 7253 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.299913 5829 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.300309 7864 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.301918 6095 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.302345 7987 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.302901 7864 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.303226 5563 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.304720 7170 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.305181 7170 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.305433 5430 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.305531 5152 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.307157 6095 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.307341 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 1211024f45e142979fc1e5e8fff010ce with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:39.308146 5562 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.308810 5430 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.308811 5827 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.309298 6095 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.309988 5148 catalog_manager.cc:5484] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:39.311830 5296 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.312913 5960 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.315941 5694 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
W20240501 19:57:39.316073 5896 consensus_peers.cc:485] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1211024f45e142979fc1e5e8fff010ce. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:39.316800 5696 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.319443 5562 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.319654 6095 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.319662 5296 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.319794 7253 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.320017 7864 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.320140 5960 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.320243 7253 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.320791 7253 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.322191 5960 raft_consensus.cc:858] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }.
Pending config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }:
New config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
W20240501 19:57:39.323196 5960 raft_consensus.cc:1565] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 5' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }.
Pending config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }:
New config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:39.323573 5960 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
W20240501 19:57:39.324497 5629 consensus_peers.cc:485] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae -> Peer 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): Couldn't send request to peer 363f4ac4964f41a6ba0f98d1f487db83. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f0df1e5e85294f60b8de22053dd577f7. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:39.324744 5430 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.325067 7253 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.327520 5827 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.327836 7864 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.328222 7253 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.328891 5827 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.329844 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet f0df1e5e85294f60b8de22053dd577f7 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:39.330273 5562 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.330312 6095 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.330364 5827 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.331862 5960 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.331879 5430 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.332132 5147 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:39.333292 5294 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.333508 5296 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:39.336186 7969 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.337734 5296 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.339107 5296 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:39.343300 5562 consensus_queue.cc:237] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:39.344815 5827 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.344887 5296 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.344956 5430 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.345057 5960 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.345111 5696 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.345270 7932 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.345433 7931 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.345580 7150 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.345718 7926 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.345888 7154 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.346098 6095 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.346490 7154 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.348384 7932 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.349053 6095 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.349220 5296 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.349323 5827 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.350975 5960 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.351811 5147 catalog_manager.cc:5484] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:39.352226 5696 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:39.353979 5430 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
W20240501 19:57:39.354738 7150 consensus_queue.cc:1159] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9544b0e566c54a62a6135f3bff47c4f2" responder_term: 1 status { last_received { term: 1 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 4 } }
I20240501 19:57:39.356998 5562 consensus_queue.cc:237] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.358654 5296 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.358814 6095 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240501 19:57:39.358929 5497 consensus_peers.cc:485] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6b6181bae544db0b6bbdc6bcd01a40d. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:39.359139 5827 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.359191 7150 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.359498 5430 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.359512 7150 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.359596 5960 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.359854 7933 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.360081 7154 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.360208 7933 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.362326 5696 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.362509 7933 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.362946 5827 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.363370 7150 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.364305 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet d6b6181bae544db0b6bbdc6bcd01a40d with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:39.365172 6095 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.365336 5296 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.365615 5154 catalog_manager.cc:5484] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:39.366540 5960 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.367273 5696 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.370069 5430 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.374109 5696 consensus_queue.cc:237] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:39.375538 5562 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.375607 6095 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.375627 5430 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.375949 7986 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.376108 7999 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.376224 7988 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.376747 5960 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.377120 5296 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.377343 7999 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.377496 7986 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.379146 5827 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:39.379343 7986 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.379670 5296 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.379922 7987 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:39.381072 5960 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.381148 5430 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.381426 5562 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.383414 5148 catalog_manager.cc:5484] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.383913 6095 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.384406 5827 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:39.388557 5696 consensus_queue.cc:237] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.390216 5827 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.390575 5296 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.390728 5430 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.390851 5960 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.390882 6095 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.391091 7987 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.391155 5562 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:39.391489 7999 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.391901 7999 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.392030 7989 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:39.392354 7989 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20240501 19:57:39.393285 7253 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240501 19:57:39.395227 5629 consensus_peers.cc:485] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bd8a8b5e92c74598a5399fe0ca8c048d. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:39.395437 7253 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.397253 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet bd8a8b5e92c74598a5399fe0ca8c048d with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:39.397222 5430 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.397610 5563 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.397792 5154 catalog_manager.cc:5484] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:39.399626 6095 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.400295 5296 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.400292 5960 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.402796 5827 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:39.460597 7730 heartbeater.cc:503] Master 127.20.210.62:45969 was elected leader, sending a full tablet report...
I20240501 19:57:39.582449 8067 ts_tablet_manager.cc:902] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83: Initiating tablet copy from peer 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499)
I20240501 19:57:39.583171 8067 tablet_copy_client.cc:308] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.5:39499
I20240501 19:57:39.586524 5849 tablet_copy_service.cc:140] P 1865147cd5a346ad8549f18fa8cc26c3: Received BeginTabletCopySession request for tablet d8a84349ba37496289351ac689f77eab from peer 363f4ac4964f41a6ba0f98d1f487db83 ({username='slave'} at 127.20.210.8:59559)
I20240501 19:57:39.586680 5849 tablet_copy_service.cc:161] P 1865147cd5a346ad8549f18fa8cc26c3: Beginning new tablet copy session on tablet d8a84349ba37496289351ac689f77eab from peer 363f4ac4964f41a6ba0f98d1f487db83 at {username='slave'} at 127.20.210.8:59559: session id = 363f4ac4964f41a6ba0f98d1f487db83-d8a84349ba37496289351ac689f77eab
I20240501 19:57:39.587853 5849 tablet_copy_source_session.cc:205] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:39.590853 8067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8a84349ba37496289351ac689f77eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:39.595572 8067 tablet_copy_client.cc:783] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:39.595744 8067 tablet_copy_client.cc:647] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:39.597213 8067 tablet_copy_client.cc:523] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:39.598592 8067 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap starting.
I20240501 19:57:39.603735 8067 log.cc:824] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:39.624380 8067 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:39.624651 8067 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap complete.
I20240501 19:57:39.624912 8067 ts_tablet_manager.cc:1367] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83: Time spent bootstrapping tablet: real 0.026s user 0.023s sys 0.000s
I20240501 19:57:39.626791 8067 raft_consensus.cc:348] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:39.627012 8067 raft_consensus.cc:729] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 363f4ac4964f41a6ba0f98d1f487db83, State: Initialized, Role: LEARNER
I20240501 19:57:39.627300 8067 consensus_queue.cc:260] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:39.628070 8067 ts_tablet_manager.cc:1398] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20240501 19:57:39.628655 5849 tablet_copy_service.cc:342] P 1865147cd5a346ad8549f18fa8cc26c3: Request end of tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-d8a84349ba37496289351ac689f77eab received from {username='slave'} at 127.20.210.8:59559
I20240501 19:57:39.628753 5849 tablet_copy_service.cc:433] P 1865147cd5a346ad8549f18fa8cc26c3: ending tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-d8a84349ba37496289351ac689f77eab on tablet d8a84349ba37496289351ac689f77eab with peer 363f4ac4964f41a6ba0f98d1f487db83
I20240501 19:57:39.651245 8081 ts_tablet_manager.cc:902] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285)
I20240501 19:57:39.651924 8081 tablet_copy_client.cc:308] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:46285
I20240501 19:57:39.655643 5317 tablet_copy_service.cc:140] P ad52038f1d38489ea464a8fbc72daac0: Received BeginTabletCopySession request for tablet 913563ed5f904cc28ff15148d9a5e5a2 from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:60929)
I20240501 19:57:39.655758 5317 tablet_copy_service.cc:161] P ad52038f1d38489ea464a8fbc72daac0: Beginning new tablet copy session on tablet 913563ed5f904cc28ff15148d9a5e5a2 from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:60929: session id = 16737daa1d234f35826143a1aa29dc0b-913563ed5f904cc28ff15148d9a5e5a2
I20240501 19:57:39.657127 5317 tablet_copy_source_session.cc:205] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:39.661808 8081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 913563ed5f904cc28ff15148d9a5e5a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:39.667747 8081 tablet_copy_client.cc:783] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:39.667928 8081 tablet_copy_client.cc:647] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:39.669543 8081 tablet_copy_client.cc:523] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:39.671104 8081 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:39.676934 8081 log.cc:824] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:39.680946 8108 ts_tablet_manager.cc:902] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085)
I20240501 19:57:39.681624 8108 tablet_copy_client.cc:308] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.6:34085
I20240501 19:57:39.687033 5983 tablet_copy_service.cc:140] P 9544b0e566c54a62a6135f3bff47c4f2: Received BeginTabletCopySession request for tablet 1211024f45e142979fc1e5e8fff010ce from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:53599)
I20240501 19:57:39.687160 5983 tablet_copy_service.cc:161] P 9544b0e566c54a62a6135f3bff47c4f2: Beginning new tablet copy session on tablet 1211024f45e142979fc1e5e8fff010ce from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:53599: session id = 143c54da1456472dadbf1d5d65e0a6f6-1211024f45e142979fc1e5e8fff010ce
I20240501 19:57:39.688481 5983 tablet_copy_source_session.cc:205] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:39.691527 8108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1211024f45e142979fc1e5e8fff010ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:39.696483 8108 tablet_copy_client.cc:783] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:39.696689 8108 tablet_copy_client.cc:647] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:39.698706 8108 tablet_copy_client.cc:523] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:39.700310 8108 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:39.702991 8081 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:39.703277 8081 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:39.703532 8081 ts_tablet_manager.cc:1367] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.033s user 0.024s sys 0.007s
I20240501 19:57:39.705905 8081 raft_consensus.cc:348] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.706175 8081 raft_consensus.cc:729] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:39.706517 8081 consensus_queue.cc:260] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.706960 8108 log.cc:824] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6: Log is configured to *not* fsync() on all Append() calls
I20240501 19:57:39.707481 8081 ts_tablet_manager.cc:1398] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240501 19:57:39.708175 5317 tablet_copy_service.cc:342] P ad52038f1d38489ea464a8fbc72daac0: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-913563ed5f904cc28ff15148d9a5e5a2 received from {username='slave'} at 127.20.210.10:60929
I20240501 19:57:39.708263 5317 tablet_copy_service.cc:433] P ad52038f1d38489ea464a8fbc72daac0: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-913563ed5f904cc28ff15148d9a5e5a2 on tablet 913563ed5f904cc28ff15148d9a5e5a2 with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:39.710366 8081 ts_tablet_manager.cc:902] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:39.710721 8081 tablet_copy_client.cc:308] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:39.714754 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet 9ecbb3dec08047478ded9809f3a0b859 from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:45619)
I20240501 19:57:39.714875 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet 9ecbb3dec08047478ded9809f3a0b859 from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:45619: session id = 16737daa1d234f35826143a1aa29dc0b-9ecbb3dec08047478ded9809f3a0b859
I20240501 19:57:39.715850 5583 tablet_copy_source_session.cc:205] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:39.716794 8081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ecbb3dec08047478ded9809f3a0b859. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:39.718683 8081 tablet_copy_client.cc:783] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:39.718824 8081 tablet_copy_client.cc:647] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:39.719990 8081 tablet_copy_client.cc:523] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:39.721107 8081 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:39.723498 8130 ts_tablet_manager.cc:902] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:39.724045 8130 tablet_copy_client.cc:308] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:39.724467 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet d3937a2d841440d793ba2637f1af060f from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:45619)
I20240501 19:57:39.724556 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet d3937a2d841440d793ba2637f1af060f from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:45619: session id = 16737daa1d234f35826143a1aa29dc0b-d3937a2d841440d793ba2637f1af060f
I20240501 19:57:39.725291 5583 tablet_copy_source_session.cc:205] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:39.725966 8147 ts_tablet_manager.cc:902] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285)
I20240501 19:57:39.725971 8130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3937a2d841440d793ba2637f1af060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:39.726583 8147 tablet_copy_client.cc:308] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:46285
I20240501 19:57:39.728672 8130 tablet_copy_client.cc:783] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:39.728863 8130 tablet_copy_client.cc:647] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:39.730913 5317 tablet_copy_service.cc:140] P ad52038f1d38489ea464a8fbc72daac0: Received BeginTabletCopySession request for tablet 95e8db3b5d81468098c01c32c4d22e80 from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:56127)
I20240501 19:57:39.731002 5317 tablet_copy_service.cc:161] P ad52038f1d38489ea464a8fbc72daac0: Beginning new tablet copy session on tablet 95e8db3b5d81468098c01c32c4d22e80 from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:56127: session id = 143c54da1456472dadbf1d5d65e0a6f6-95e8db3b5d81468098c01c32c4d22e80
I20240501 19:57:39.731626 8130 tablet_copy_client.cc:523] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:39.731693 5317 tablet_copy_source_session.cc:205] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:39.732386 8147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95e8db3b5d81468098c01c32c4d22e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:39.732657 8130 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:39.734472 8147 tablet_copy_client.cc:783] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:39.734637 8147 tablet_copy_client.cc:647] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:39.735697 8147 tablet_copy_client.cc:523] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:39.736828 8147 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:39.738373 8108 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:39.738657 8108 tablet_bootstrap.cc:492] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:39.738912 8108 ts_tablet_manager.cc:1367] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.039s user 0.031s sys 0.004s
I20240501 19:57:39.741145 8108 raft_consensus.cc:348] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:39.741415 8108 raft_consensus.cc:729] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:39.741756 8108 consensus_queue.cc:260] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:39.742583 8108 ts_tablet_manager.cc:1398] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240501 19:57:39.745316 5983 tablet_copy_service.cc:342] P 9544b0e566c54a62a6135f3bff47c4f2: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-1211024f45e142979fc1e5e8fff010ce received from {username='slave'} at 127.20.210.9:53599
I20240501 19:57:39.745398 5983 tablet_copy_service.cc:433] P 9544b0e566c54a62a6135f3bff47c4f2: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-1211024f45e142979fc1e5e8fff010ce on tablet 1211024f45e142979fc1e5e8fff010ce with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:39.751204 8081 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:39.751435 8081 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:39.751586 8081 ts_tablet_manager.cc:1367] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.031s user 0.020s sys 0.003s
I20240501 19:57:39.752072 8081 raft_consensus.cc:348] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.752200 8081 raft_consensus.cc:729] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:39.752307 8081 consensus_queue.cc:260] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.753691 8081 ts_tablet_manager.cc:1398] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240501 19:57:39.754153 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-9ecbb3dec08047478ded9809f3a0b859 received from {username='slave'} at 127.20.210.10:45619
I20240501 19:57:39.754276 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-9ecbb3dec08047478ded9809f3a0b859 on tablet 9ecbb3dec08047478ded9809f3a0b859 with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:39.767900 8147 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:39.768173 8147 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:39.768321 8147 ts_tablet_manager.cc:1367] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.032s user 0.024s sys 0.001s
I20240501 19:57:39.768810 8147 raft_consensus.cc:348] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:39.768950 8147 raft_consensus.cc:729] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:39.769048 8147 consensus_queue.cc:260] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:39.769315 8147 ts_tablet_manager.cc:1398] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:39.769801 5317 tablet_copy_service.cc:342] P ad52038f1d38489ea464a8fbc72daac0: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-95e8db3b5d81468098c01c32c4d22e80 received from {username='slave'} at 127.20.210.9:56127
I20240501 19:57:39.769870 5317 tablet_copy_service.cc:433] P ad52038f1d38489ea464a8fbc72daac0: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-95e8db3b5d81468098c01c32c4d22e80 on tablet 95e8db3b5d81468098c01c32c4d22e80 with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:39.773381 8130 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:39.773655 8130 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:39.773829 8130 ts_tablet_manager.cc:1367] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.041s user 0.020s sys 0.004s
I20240501 19:57:39.774214 8130 raft_consensus.cc:348] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.774371 8130 raft_consensus.cc:729] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:39.774470 8130 consensus_queue.cc:260] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.774684 8130 ts_tablet_manager.cc:1398] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240501 19:57:39.775287 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-d3937a2d841440d793ba2637f1af060f received from {username='slave'} at 127.20.210.10:45619
I20240501 19:57:39.775374 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-d3937a2d841440d793ba2637f1af060f on tablet d3937a2d841440d793ba2637f1af060f with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:39.792938 8067 ts_tablet_manager.cc:902] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:39.793368 8067 tablet_copy_client.cc:308] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:39.796881 5716 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet f0df1e5e85294f60b8de22053dd577f7 from peer 363f4ac4964f41a6ba0f98d1f487db83 ({username='slave'} at 127.20.210.8:53017)
I20240501 19:57:39.797013 5716 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet f0df1e5e85294f60b8de22053dd577f7 from peer 363f4ac4964f41a6ba0f98d1f487db83 at {username='slave'} at 127.20.210.8:53017: session id = 363f4ac4964f41a6ba0f98d1f487db83-f0df1e5e85294f60b8de22053dd577f7
I20240501 19:57:39.798080 5716 tablet_copy_source_session.cc:205] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:39.799190 8067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0df1e5e85294f60b8de22053dd577f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:39.801249 8067 tablet_copy_client.cc:783] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:39.801409 8067 tablet_copy_client.cc:647] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:39.802636 8067 tablet_copy_client.cc:523] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:39.803656 8067 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap starting.
I20240501 19:57:39.822402 8067 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:39.822623 8067 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap complete.
I20240501 19:57:39.822772 8067 ts_tablet_manager.cc:1367] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent bootstrapping tablet: real 0.019s user 0.017s sys 0.003s
I20240501 19:57:39.823171 8067 raft_consensus.cc:348] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:39.823305 8067 raft_consensus.cc:729] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 363f4ac4964f41a6ba0f98d1f487db83, State: Initialized, Role: LEARNER
I20240501 19:57:39.823412 8067 consensus_queue.cc:260] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:39.823678 8067 ts_tablet_manager.cc:1398] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:39.824156 5716 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-f0df1e5e85294f60b8de22053dd577f7 received from {username='slave'} at 127.20.210.8:53017
I20240501 19:57:39.824229 5716 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-f0df1e5e85294f60b8de22053dd577f7 on tablet f0df1e5e85294f60b8de22053dd577f7 with peer 363f4ac4964f41a6ba0f98d1f487db83
I20240501 19:57:39.838199 8130 ts_tablet_manager.cc:902] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:39.838608 8130 tablet_copy_client.cc:308] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:39.838992 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet d6b6181bae544db0b6bbdc6bcd01a40d from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:45619)
I20240501 19:57:39.839080 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet d6b6181bae544db0b6bbdc6bcd01a40d from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:45619: session id = 16737daa1d234f35826143a1aa29dc0b-d6b6181bae544db0b6bbdc6bcd01a40d
I20240501 19:57:39.839752 5583 tablet_copy_source_session.cc:205] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:39.840478 8130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b6181bae544db0b6bbdc6bcd01a40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:39.842123 8130 tablet_copy_client.cc:783] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:39.842288 8130 tablet_copy_client.cc:647] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:39.843226 8130 tablet_copy_client.cc:523] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:39.844033 8130 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:39.860605 8130 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:39.860810 8130 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:39.860926 8130 ts_tablet_manager.cc:1367] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.003s
I20240501 19:57:39.861241 8130 raft_consensus.cc:348] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.861335 8130 raft_consensus.cc:729] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:39.861402 8130 consensus_queue.cc:260] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.861605 8130 ts_tablet_manager.cc:1398] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:39.862108 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-d6b6181bae544db0b6bbdc6bcd01a40d received from {username='slave'} at 127.20.210.10:45619
I20240501 19:57:39.862179 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-d6b6181bae544db0b6bbdc6bcd01a40d on tablet d6b6181bae544db0b6bbdc6bcd01a40d with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:39.925869 8130 ts_tablet_manager.cc:902] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:39.926283 8130 tablet_copy_client.cc:308] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:39.929664 5716 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet bd8a8b5e92c74598a5399fe0ca8c048d from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:34389)
I20240501 19:57:39.929765 5716 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet bd8a8b5e92c74598a5399fe0ca8c048d from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:34389: session id = 16737daa1d234f35826143a1aa29dc0b-bd8a8b5e92c74598a5399fe0ca8c048d
I20240501 19:57:39.930694 5716 tablet_copy_source_session.cc:205] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:39.931862 8130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd8a8b5e92c74598a5399fe0ca8c048d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:39.933671 8130 tablet_copy_client.cc:783] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:39.933830 8130 tablet_copy_client.cc:647] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:39.934804 8130 tablet_copy_client.cc:523] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:39.935575 8130 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:39.958145 8130 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:39.958391 8130 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:39.958528 8130 ts_tablet_manager.cc:1367] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.023s user 0.018s sys 0.005s
I20240501 19:57:39.958866 8130 raft_consensus.cc:348] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.958978 8130 raft_consensus.cc:729] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:39.959053 8130 consensus_queue.cc:260] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:39.959534 8130 ts_tablet_manager.cc:1398] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:39.959932 5716 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-bd8a8b5e92c74598a5399fe0ca8c048d received from {username='slave'} at 127.20.210.10:34389
I20240501 19:57:39.960000 5716 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-bd8a8b5e92c74598a5399fe0ca8c048d on tablet bd8a8b5e92c74598a5399fe0ca8c048d with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:40.003203 7418 raft_consensus.cc:1203] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:40.037992 7684 raft_consensus.cc:1203] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:40.059526 7550 raft_consensus.cc:1203] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:40.122515 7684 raft_consensus.cc:1203] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:40.214210 7418 raft_consensus.cc:1203] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:40.226060 7684 raft_consensus.cc:1203] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:40.266110 7550 raft_consensus.cc:1203] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:40.302834 7684 raft_consensus.cc:1203] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:40.405156 7684 raft_consensus.cc:1203] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:40.511744 7912 raft_consensus.cc:1050] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:40.512218 7912 consensus_queue.cc:237] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:40.513823 5430 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.513911 5563 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.513908 5827 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.513962 7684 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.514528 8306 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.514714 7912 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.514887 6095 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.515026 8308 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.515206 8307 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.515476 7912 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.515884 5960 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.516369 5694 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.517086 7912 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.517364 7912 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.519021 7912 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.519798 6095 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.519891 5960 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.521371 5430 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.521819 5563 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.522126 5827 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.522331 5694 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.523304 5153 catalog_manager.cc:5484] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:40.527063 7970 raft_consensus.cc:1050] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:40.527606 7970 consensus_queue.cc:237] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:40.528617 7684 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.529714 5430 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.529754 5696 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.530181 7967 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.530301 5563 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.530438 8322 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.530833 8322 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.532132 5296 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.532732 5827 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.533918 5296 consensus_queue.cc:237] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:40.535161 7550 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.535578 7684 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.535730 5430 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.535718 5827 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.536077 8307 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.537094 8307 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.537385 8307 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.538635 6097 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.538682 5960 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.538700 7967 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.538960 8306 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.539036 5563 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.539017 7967 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.539315 7967 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.539355 8306 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.539633 8306 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.541205 8306 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.541947 7967 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.542268 5563 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.543915 5960 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.544642 5429 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.544936 5150 catalog_manager.cc:5484] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:40.545284 5148 catalog_manager.cc:5484] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.545480 5294 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.545668 5696 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.546921 5430 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.547017 7550 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.548815 7684 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.548909 5563 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.549360 5138 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 913563ed5f904cc28ff15148d9a5e5a2 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:40.550407 5829 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.552577 5674 tablet_service.cc:1514] Processing DeleteTablet for tablet 913563ed5f904cc28ff15148d9a5e5a2 with delete_type TABLET_DATA_TOMBSTONED (TS 4d4855c096ac4e1e936abf15834332ae not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:36532
I20240501 19:57:40.552947 7934 raft_consensus.cc:1050] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:40.553390 7934 consensus_queue.cc:237] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:40.554886 6095 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.555011 7684 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.555361 8339 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.556232 5962 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.557037 8339 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.558339 5828 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.558311 8323 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.558848 8335 tablet_replica.cc:330] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae: stopping tablet replica
I20240501 19:57:40.559092 8335 raft_consensus.cc:2229] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:40.559270 8335 raft_consensus.cc:2258] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:40.559334 6097 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.560258 8335 ts_tablet_manager.cc:1875] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:40.561283 5430 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.561693 5294 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.563050 8335 ts_tablet_manager.cc:1888] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:40.563170 8335 log.cc:1196] T 913563ed5f904cc28ff15148d9a5e5a2 P 4d4855c096ac4e1e936abf15834332ae: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/913563ed5f904cc28ff15148d9a5e5a2
I20240501 19:57:40.563112 8343 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.563212 6096 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.563629 8343 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.564693 8339 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.565536 5141 catalog_manager.cc:4830] TS 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519): tablet 913563ed5f904cc28ff15148d9a5e5a2 (table rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]) successfully deleted
I20240501 19:57:40.568437 5829 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.569133 5694 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.569499 8343 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.569521 6095 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.570533 8340 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.570966 8343 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.571731 5430 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.571822 5960 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.572952 5294 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.573559 5829 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.575011 5147 catalog_manager.cc:5484] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.576941 7684 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.577874 6095 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.582197 5694 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.584000 5563 consensus_queue.cc:237] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:40.585496 5294 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.585672 5430 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.586033 5694 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.586050 8338 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.586122 6095 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.586169 8341 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.586478 8341 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.586568 5960 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.586592 8343 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.586894 8338 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.587703 7684 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.588001 8338 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.589042 8343 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.589397 5694 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.590781 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d3937a2d841440d793ba2637f1af060f with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:40.591084 5430 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.591082 7684 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.591296 5294 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.593557 5145 catalog_manager.cc:5484] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b reported cstate change: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.594602 5960 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.594659 6095 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.597775 5809 tablet_service.cc:1514] Processing DeleteTablet for tablet d3937a2d841440d793ba2637f1af060f with delete_type TABLET_DATA_TOMBSTONED (TS 1865147cd5a346ad8549f18fa8cc26c3 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:35230
I20240501 19:57:40.598945 8352 tablet_replica.cc:330] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3: stopping tablet replica
I20240501 19:57:40.599259 8352 raft_consensus.cc:2229] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:40.599479 8352 raft_consensus.cc:2258] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:40.600612 8352 ts_tablet_manager.cc:1875] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:40.602589 8352 ts_tablet_manager.cc:1888] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:40.602671 8352 log.cc:1196] T d3937a2d841440d793ba2637f1af060f P 1865147cd5a346ad8549f18fa8cc26c3: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/wals/d3937a2d841440d793ba2637f1af060f
I20240501 19:57:40.603220 5139 catalog_manager.cc:4830] TS 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499): tablet d3937a2d841440d793ba2637f1af060f (table rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]) successfully deleted
I20240501 19:57:40.626641 8075 raft_consensus.cc:1050] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3: attempting to promote NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 to VOTER
I20240501 19:57:40.627172 8075 consensus_queue.cc:237] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:40.628826 7418 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.628871 6095 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.629283 8359 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.629410 5960 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.629434 5430 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.629498 8356 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.629734 5694 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.629787 8360 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.630170 5294 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.630735 8356 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.631088 8359 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.631412 8359 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.631641 5563 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.633208 8360 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.633809 8359 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.634428 5294 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.634496 5430 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.636039 6095 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.636284 5960 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.637200 7418 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.638463 5147 catalog_manager.cc:5484] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.641896 5694 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.646292 5563 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.648281 5829 consensus_queue.cc:237] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:40.649737 6095 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.650130 7418 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.650202 8357 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.650861 8358 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.652143 5694 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.652303 5294 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.652458 5430 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.652669 8357 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.652778 8359 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.653000 5563 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.653312 8357 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.653626 8358 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.655205 8359 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.655952 7418 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.656728 5139 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d8a84349ba37496289351ac689f77eab with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:40.657670 5563 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.658154 6095 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.658188 5430 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.657281 5294 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.659034 5152 catalog_manager.cc:5484] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 reported cstate change: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.661162 5694 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.665609 5942 tablet_service.cc:1514] Processing DeleteTablet for tablet d8a84349ba37496289351ac689f77eab with delete_type TABLET_DATA_TOMBSTONED (TS 9544b0e566c54a62a6135f3bff47c4f2 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:42138
I20240501 19:57:40.666126 8372 tablet_replica.cc:330] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: stopping tablet replica
I20240501 19:57:40.666661 8372 raft_consensus.cc:2229] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:40.666922 8372 raft_consensus.cc:2258] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:40.668359 8372 ts_tablet_manager.cc:1875] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:40.670400 8372 ts_tablet_manager.cc:1888] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:40.670512 8372 log.cc:1196] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/wals/d8a84349ba37496289351ac689f77eab
I20240501 19:57:40.671039 5141 catalog_manager.cc:4830] TS 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085): tablet d8a84349ba37496289351ac689f77eab (table rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]) successfully deleted
I20240501 19:57:40.715030 7987 raft_consensus.cc:1050] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 to VOTER
I20240501 19:57:40.715409 7987 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:40.716954 5563 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.717097 5294 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.717130 6095 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.717411 5960 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.717423 5829 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.717553 8417 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.717687 7418 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.717737 7253 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.717892 8415 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.717972 5430 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.718045 7989 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.718287 7253 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.718449 8415 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.718595 8417 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.721086 8416 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.721675 6095 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.721701 7418 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.722666 5960 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.724227 5152 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 reported cstate change: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.724604 5294 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.724953 5829 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.726306 5430 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.731614 5563 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.734084 5694 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:40.735378 5563 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.735611 5430 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.735611 7418 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.735620 5294 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.735894 6095 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.735953 8415 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.736017 5829 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.736135 7987 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.736229 8415 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.736368 8417 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.736488 7989 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.736748 7253 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.738225 8417 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.738730 5430 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.739523 7418 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.740911 6095 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.741032 5563 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.741216 5152 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.742280 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f0df1e5e85294f60b8de22053dd577f7 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:40.743093 5294 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.744375 5829 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:40.745626 5942 tablet_service.cc:1514] Processing DeleteTablet for tablet f0df1e5e85294f60b8de22053dd577f7 with delete_type TABLET_DATA_TOMBSTONED (TS 9544b0e566c54a62a6135f3bff47c4f2 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:42138
I20240501 19:57:40.745819 8372 tablet_replica.cc:330] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2: stopping tablet replica
I20240501 19:57:40.745949 8372 raft_consensus.cc:2229] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:40.746138 8372 raft_consensus.cc:2258] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:40.746960 8372 ts_tablet_manager.cc:1875] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:40.749109 8372 ts_tablet_manager.cc:1888] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:40.749181 8372 log.cc:1196] T f0df1e5e85294f60b8de22053dd577f7 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/wals/f0df1e5e85294f60b8de22053dd577f7
I20240501 19:57:40.749801 5141 catalog_manager.cc:4830] TS 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085): tablet f0df1e5e85294f60b8de22053dd577f7 (table rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]) successfully deleted
I20240501 19:57:40.750012 8340 raft_consensus.cc:1050] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:40.750469 8340 consensus_queue.cc:237] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:40.752169 5430 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.752229 5960 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.752306 6095 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.752394 7684 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.752624 5694 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.752665 8338 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.752872 5829 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.752924 8343 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.753083 8339 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.753396 8343 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.753567 8342 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.754755 8343 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.754843 5294 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.755401 8343 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.757696 8342 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.758116 5294 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.759749 5430 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.760545 7684 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.760545 5829 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.760993 5152 catalog_manager.cc:5484] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.762773 6097 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.764153 5694 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.766686 5960 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.767678 8340 raft_consensus.cc:1050] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:40.768216 8340 consensus_queue.cc:237] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:40.768631 5563 consensus_queue.cc:237] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:40.770264 5829 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.770305 5827 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.770655 8340 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.770848 7683 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.770689 8343 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.771451 5294 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.771595 5429 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.772035 8341 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.772248 8341 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.772462 8341 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.772769 7684 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.773167 8338 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.773625 5296 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.774504 5430 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.774873 8340 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.775002 8343 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.775732 5962 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.775846 5960 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.776129 5694 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.776156 8338 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.776723 8343 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.776837 8340 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.777289 6095 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.778008 6097 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.779409 8338 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.779383 8343 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.779893 8343 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.781131 5960 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.781481 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9ecbb3dec08047478ded9809f3a0b859 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:40.781860 7683 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.783360 5294 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.784289 6095 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.784601 5152 catalog_manager.cc:5484] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.785779 8343 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.786159 7684 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.787109 5674 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ecbb3dec08047478ded9809f3a0b859 with delete_type TABLET_DATA_TOMBSTONED (TS 4d4855c096ac4e1e936abf15834332ae not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:36532
I20240501 19:57:40.787359 8335 tablet_replica.cc:330] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: stopping tablet replica
I20240501 19:57:40.787477 8307 raft_consensus.cc:1050] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:40.787492 8335 raft_consensus.cc:2229] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:40.787658 8335 raft_consensus.cc:2258] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:40.787992 8307 consensus_queue.cc:237] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:40.788681 8335 ts_tablet_manager.cc:1875] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:40.789633 5829 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.790313 7550 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.790311 5827 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.790738 8306 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.791265 5962 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.791265 5563 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.791535 6095 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.791738 5694 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.792397 5426 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:40.792709 8306 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.792765 8335 ts_tablet_manager.cc:1888] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:40.792829 8335 log.cc:1196] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/9ecbb3dec08047478ded9809f3a0b859
I20240501 19:57:40.792829 8308 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.793401 5426 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.793450 5141 catalog_manager.cc:4830] TS 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519): tablet 9ecbb3dec08047478ded9809f3a0b859 (table rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]) successfully deleted
I20240501 19:57:40.794075 7915 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.794404 7915 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.794730 7915 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.795037 6906 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:40.795500 5152 catalog_manager.cc:5484] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b reported cstate change: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.796094 6097 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.798813 5827 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.799433 5296 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.799912 5960 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.800189 5696 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.801609 5426 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.804925 8443 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.805822 5429 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.807281 5829 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.807984 5562 consensus_queue.cc:237] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:40.808560 7550 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.809461 5152 catalog_manager.cc:5484] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.809708 5429 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.809993 7684 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.810302 8339 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.810961 5960 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.810992 5827 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.811265 6095 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.811316 8340 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.811664 8340 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.812511 8338 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.812495 5960 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.812963 8340 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.812906 6095 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.814312 5296 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.816058 8338 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.816617 5694 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.817080 8339 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.818394 5562 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.818630 6095 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.819096 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d6b6181bae544db0b6bbdc6bcd01a40d with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:40.819193 5429 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.819535 7684 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.822402 5962 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.822693 5145 catalog_manager.cc:5484] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.824122 5829 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.824422 5294 consensus_queue.cc:237] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:40.824858 5674 tablet_service.cc:1514] Processing DeleteTablet for tablet d6b6181bae544db0b6bbdc6bcd01a40d with delete_type TABLET_DATA_TOMBSTONED (TS 4d4855c096ac4e1e936abf15834332ae not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:36532
I20240501 19:57:40.825064 8335 tablet_replica.cc:330] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae: stopping tablet replica
I20240501 19:57:40.825171 8335 raft_consensus.cc:2229] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:40.825328 8335 raft_consensus.cc:2258] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:40.826308 8335 ts_tablet_manager.cc:1875] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:40.826709 7550 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.827000 8306 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.827951 5829 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.828096 8335 ts_tablet_manager.cc:1888] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:40.828157 8335 log.cc:1196] T d6b6181bae544db0b6bbdc6bcd01a40d P 4d4855c096ac4e1e936abf15834332ae: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/d6b6181bae544db0b6bbdc6bcd01a40d
I20240501 19:57:40.828274 8306 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.828706 5141 catalog_manager.cc:4830] TS 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519): tablet d6b6181bae544db0b6bbdc6bcd01a40d (table rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]) successfully deleted
I20240501 19:57:40.829088 5562 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.829178 5694 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.829306 6095 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.830796 8306 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.830941 7915 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.831095 8308 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.832862 5296 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:40.835345 5429 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:40.835705 7915 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:40.836570 8306 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.837127 5562 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.837828 5138 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 95e8db3b5d81468098c01c32c4d22e80 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:40.838544 5429 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.839295 7550 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.839330 5694 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.839290 5829 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.841447 5147 catalog_manager.cc:5484] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.845525 5942 tablet_service.cc:1514] Processing DeleteTablet for tablet 95e8db3b5d81468098c01c32c4d22e80 with delete_type TABLET_DATA_TOMBSTONED (TS 9544b0e566c54a62a6135f3bff47c4f2 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:42138
I20240501 19:57:40.845698 8372 tablet_replica.cc:330] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2: stopping tablet replica
I20240501 19:57:40.845796 8372 raft_consensus.cc:2229] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:40.845901 8372 raft_consensus.cc:2258] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:40.846626 8372 ts_tablet_manager.cc:1875] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:40.847987 8372 ts_tablet_manager.cc:1888] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:40.848052 8372 log.cc:1196] T 95e8db3b5d81468098c01c32c4d22e80 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/wals/95e8db3b5d81468098c01c32c4d22e80
I20240501 19:57:40.848467 5141 catalog_manager.cc:4830] TS 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085): tablet 95e8db3b5d81468098c01c32c4d22e80 (table rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]) successfully deleted
I20240501 19:57:40.840910 6095 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:40.915900 5694 consensus_queue.cc:237] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:40.917301 5829 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.917351 5429 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.917409 5296 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.917655 7253 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.917697 6095 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.917743 5962 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.917881 8415 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.918017 5562 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.918038 7253 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.918674 7987 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.919028 7987 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.920575 8415 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.921100 7253 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.922142 5296 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.924007 5145 catalog_manager.cc:5484] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:40.924793 5829 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.927525 5694 consensus_queue.cc:237] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:40.927752 6095 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.929497 6095 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:40.929756 5829 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:40.929916 5960 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:40.930279 8417 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:40.930364 7989 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.930828 7989 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:40.930951 5962 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.933698 5296 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240501 19:57:40.934458 5629 consensus_peers.cc:485] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 064f698b376e407da78c7d439553896a. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:40.934597 8415 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:40.934698 5563 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:40.934871 7253 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:40.935184 5426 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:40.935441 8415 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.936525 5563 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.937891 5563 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:40.938489 5829 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:40.940614 5150 catalog_manager.cc:5484] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:40.940788 8417 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.941304 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 064f698b376e407da78c7d439553896a with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:40.941289 5296 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:40.941697 5426 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.942417 6095 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:40.942656 5960 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:40.943015 5426 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:40.959039 5694 consensus_queue.cc:237] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:40.960434 5960 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.960517 6095 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.960896 5563 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.960965 5829 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.961328 8464 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.961486 7989 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.961549 5296 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.961786 7989 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.961917 8417 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.962030 8416 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.962616 5426 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.963527 7989 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.964293 7989 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.965106 5960 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.965608 6095 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.966588 5296 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.967434 5150 catalog_manager.cc:5484] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.968976 5563 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.969734 5829 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.971982 5426 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.973268 5694 consensus_queue.cc:237] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
W20240501 19:57:40.975786 5629 consensus_peers.cc:485] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae -> Peer 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): Couldn't send request to peer 363f4ac4964f41a6ba0f98d1f487db83. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f27a79cf3a044383918f5d6f600a0fd9. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:40.975869 5296 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:40.975869 5829 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:40.976212 8417 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:40.976372 6095 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:40.976418 8416 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:40.976465 5563 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:40.976809 5960 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:40.977262 8416 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:40.977406 8417 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:40.977718 8464 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:40.978794 5426 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:40.979269 8416 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:40.980365 5962 consensus_queue.cc:237] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:40.980618 8464 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:40.981319 6095 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:40.982674 5429 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.982806 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet f27a79cf3a044383918f5d6f600a0fd9 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:40.982806 5296 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:40.983333 8321 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.984006 5145 catalog_manager.cc:5484] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:40.984196 5295 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.984740 5827 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:40.985028 6095 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.985532 7970 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.985821 7970 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.986955 5829 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.987918 8321 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.988104 5694 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.988884 8321 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.988799 5960 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:40.989464 5562 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:40.992036 5426 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:40.992638 7970 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.992976 5563 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:40.995126 7969 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:40.995573 5829 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.996711 5563 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.997603 5426 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.997818 5150 catalog_manager.cc:5484] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:40.998500 6097 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:40.998999 5296 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:41.000010 5696 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:41.004173 5960 consensus_queue.cc:237] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.005920 5296 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.006402 7969 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.006953 5829 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.006953 5426 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.007833 5696 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.008020 5563 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.008073 8323 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.008133 6095 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.008425 8323 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.009665 8323 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.009850 7969 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.010192 8325 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.011840 8323 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.012506 5426 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.013159 6095 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.014024 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 2f89a8073ad64580b7aa8612204b3d38 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:41.014442 5296 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.015127 5150 catalog_manager.cc:5484] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.015532 5563 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.015702 5829 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.017477 5694 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
W20240501 19:57:41.020290 5896 consensus_peers.cc:485] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 -> Peer 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): Couldn't send request to peer 363f4ac4964f41a6ba0f98d1f487db83. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f89a8073ad64580b7aa8612204b3d38. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:41.027088 7253 raft_consensus.cc:1050] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:41.027504 7253 consensus_queue.cc:237] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:41.029040 5563 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:41.029072 5426 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:41.029072 6095 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:41.029543 5829 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:41.029543 5296 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:41.029659 7987 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:41.029882 8415 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:41.029935 5960 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:41.030014 7987 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:41.030174 8464 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:41.030372 8416 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:41.030524 8417 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:41.031985 7684 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:41.032387 8416 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:41.033226 8416 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.033658 5563 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.034626 5296 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.035609 6095 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.036101 5153 catalog_manager.cc:5484] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.036769 5829 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.038759 5426 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.039290 7684 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.040014 5960 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.047138 5694 consensus_queue.cc:237] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:41.048671 5296 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:41.048743 7684 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:41.048871 5960 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:41.048933 5563 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:41.049196 6095 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:41.049217 8415 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:41.049366 7987 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:41.049553 8417 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:41.049839 8417 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:41.049904 5426 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:41.050302 8415 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:41.051198 7987 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:41.052122 7987 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.052588 5296 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.052810 5563 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.053465 5426 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.055397 5149 catalog_manager.cc:5484] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.056238 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bd8a8b5e92c74598a5399fe0ca8c048d with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:41.056423 6095 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.057765 7684 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.058339 5960 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:41.060371 5809 tablet_service.cc:1514] Processing DeleteTablet for tablet bd8a8b5e92c74598a5399fe0ca8c048d with delete_type TABLET_DATA_TOMBSTONED (TS 1865147cd5a346ad8549f18fa8cc26c3 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:35230
I20240501 19:57:41.060583 8352 tablet_replica.cc:330] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3: stopping tablet replica
I20240501 19:57:41.060719 8352 raft_consensus.cc:2229] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:41.060881 8352 raft_consensus.cc:2258] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:41.061596 8352 ts_tablet_manager.cc:1875] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:41.063655 8352 ts_tablet_manager.cc:1888] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:41.063767 8352 log.cc:1196] T bd8a8b5e92c74598a5399fe0ca8c048d P 1865147cd5a346ad8549f18fa8cc26c3: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/wals/bd8a8b5e92c74598a5399fe0ca8c048d
I20240501 19:57:41.064327 5960 consensus_queue.cc:237] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:41.064563 5139 catalog_manager.cc:4830] TS 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499): tablet bd8a8b5e92c74598a5399fe0ca8c048d (table rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]) successfully deleted
I20240501 19:57:41.068161 7550 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:41.068336 5563 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:41.068593 6097 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:41.068719 7970 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:41.068822 5296 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:41.068822 5694 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:41.069140 7970 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:41.069537 5426 raft_consensus.cc:1261] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:41.070381 7967 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:41.070585 8321 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:41.070838 8322 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:41.071355 7967 consensus_queue.cc:1035] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:41.073112 8322 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:41.074707 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1211024f45e142979fc1e5e8fff010ce with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:41.075215 5296 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:41.075307 7550 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:41.075616 5694 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:41.075695 5154 catalog_manager.cc:5484] T 1211024f45e142979fc1e5e8fff010ce P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:41.078539 5809 tablet_service.cc:1514] Processing DeleteTablet for tablet 1211024f45e142979fc1e5e8fff010ce with delete_type TABLET_DATA_TOMBSTONED (TS 1865147cd5a346ad8549f18fa8cc26c3 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:35230
I20240501 19:57:41.078727 8352 tablet_replica.cc:330] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3: stopping tablet replica
I20240501 19:57:41.078876 8352 raft_consensus.cc:2229] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:41.079042 8352 raft_consensus.cc:2258] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:41.079056 5563 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:41.080197 8352 ts_tablet_manager.cc:1875] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:41.081815 8352 ts_tablet_manager.cc:1888] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:41.081887 8352 log.cc:1196] T 1211024f45e142979fc1e5e8fff010ce P 1865147cd5a346ad8549f18fa8cc26c3: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/wals/1211024f45e142979fc1e5e8fff010ce
I20240501 19:57:41.082000 6097 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:41.083797 5139 catalog_manager.cc:4830] TS 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499): tablet 1211024f45e142979fc1e5e8fff010ce (table rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]) successfully deleted
I20240501 19:57:41.085340 5426 raft_consensus.cc:2932] T 1211024f45e142979fc1e5e8fff010ce P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:41.309334 5829 consensus_queue.cc:237] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:41.310802 6097 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.310962 5960 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.310987 5694 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.311167 5296 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.311183 5426 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.311281 5563 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.311371 8555 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.311587 8554 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.311787 8359 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.312022 8553 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.312127 8359 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.312208 8355 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.314736 8554 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:41.315281 5694 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:41.315598 5563 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:41.315732 6097 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:41.317832 5149 catalog_manager.cc:5484] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:41.319001 5426 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:41.319344 5296 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:41.320818 5960 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:41.325875 5829 consensus_queue.cc:237] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:41.328702 5694 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.328693 5296 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.329000 6097 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.329155 5426 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.329684 8563 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.329697 5563 consensus_queue.cc:237] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:41.329905 5960 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.330657 5561 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.330821 8359 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.331224 8555 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.331817 8355 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.332087 8355 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.332284 8359 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240501 19:57:41.334069 5762 consensus_peers.cc:485] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a713bc0643d4bdf91f669dde94f63f8. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:41.334156 5829 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.334300 5296 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.334586 5962 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.334810 8340 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.334959 8553 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.335100 8343 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.335309 5694 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.335542 8340 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.336733 5139 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4a713bc0643d4bdf91f669dde94f63f8 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:41.337268 6097 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.337553 5295 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.337585 5426 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.337755 5154 catalog_manager.cc:5484] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.338402 5694 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.339588 8575 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.340086 5426 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.340428 8575 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.341261 8573 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.344113 5561 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.345916 8574 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.345881 6097 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.347586 5426 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.347759 5829 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.349392 6097 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.349579 5150 catalog_manager.cc:5484] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:41.352980 5962 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.353116 5561 consensus_queue.cc:237] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:41.353240 5296 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.356479 5962 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.356482 6097 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.356901 8343 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.357118 8343 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.357169 5962 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
W20240501 19:57:41.359999 5497 consensus_peers.cc:485] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 80fdea49202c4ab5b3972b2c9ad7cff5. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:41.360244 5296 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.360265 5694 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.360323 5829 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.360533 5426 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.360605 8340 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.360750 8571 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.360967 8571 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.361043 8573 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.362272 5694 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.362900 8340 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.364573 5694 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.365095 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 80fdea49202c4ab5b3972b2c9ad7cff5 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:41.365012 5426 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.366034 5829 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.366675 5960 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.367017 5296 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.367233 5150 catalog_manager.cc:5484] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.369587 6097 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:41.372376 5561 consensus_queue.cc:237] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:41.373821 5426 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.374140 6097 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.374130 5296 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.374154 5960 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.374154 5694 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.374485 8571 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.374626 8338 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.374755 8575 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.374891 8343 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.375002 8572 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.375108 5829 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.375440 8338 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.377466 8571 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.378095 5426 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.378666 5296 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.379437 5829 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.380371 5154 catalog_manager.cc:5484] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.382006 5694 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.382220 5960 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.384402 6097 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.386442 5561 consensus_queue.cc:237] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.388119 6097 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.388258 5694 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.388453 5829 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.388491 8343 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.388736 8573 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.388833 5426 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.389062 5296 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.389144 8573 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.389245 5960 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.389480 8573 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.391198 8343 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.392136 8343 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.393010 8575 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.393646 5426 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.393821 5827 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.395207 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet c6ba23a7b4d24e0aa077e47798e890b8 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:41.395941 5153 catalog_manager.cc:5484] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.396003 5694 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.396258 6097 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
W20240501 19:57:41.397033 5497 consensus_peers.cc:485] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): Couldn't send request to peer 363f4ac4964f41a6ba0f98d1f487db83. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c6ba23a7b4d24e0aa077e47798e890b8. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:41.397459 5296 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.399159 5829 consensus_queue.cc:237] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:41.400399 5296 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.400393 5426 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.400404 5561 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.400503 6097 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.400768 8555 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.400914 8355 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.401062 8554 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.401191 8359 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.402130 5962 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.402413 5694 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.402824 8555 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.403031 8563 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.404417 5960 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.405189 8563 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.406221 5296 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.406939 5426 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.407217 5154 catalog_manager.cc:5484] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:41.407483 5563 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.409881 6097 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.411480 5694 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.411912 5829 consensus_queue.cc:237] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
W20240501 19:57:41.414018 5762 consensus_peers.cc:485] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 -> Peer 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): Couldn't send request to peer 363f4ac4964f41a6ba0f98d1f487db83. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e17a6ae467bf42abac714a94e1ec28e5. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:41.414160 6097 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.415052 5296 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.415058 5962 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.415268 8554 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.415323 5696 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.415407 8555 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.415728 8554 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.416045 8359 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.416050 5962 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.417546 5426 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.417632 5563 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.417634 8355 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.417932 8553 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.418898 5139 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet e17a6ae467bf42abac714a94e1ec28e5 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:41.418763 5563 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.419205 8555 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.419495 5296 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.420708 6097 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.420881 5426 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.422083 5154 catalog_manager.cc:5484] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.422375 5694 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.426139 5962 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.431622 5960 consensus_queue.cc:237] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:41.432960 5563 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.433065 5829 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.433177 6097 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.433380 8325 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.433658 7969 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.433795 8321 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.434338 5296 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.434404 5694 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.434692 7969 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.434067 5426 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.435019 7969 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.435739 7969 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.436445 7969 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.437314 6097 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.437374 5694 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.438417 5563 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.438864 5296 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.439702 5147 catalog_manager.cc:5484] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:41.441071 5829 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.441589 5426 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.443795 5960 consensus_queue.cc:237] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.446173 6097 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240501 19:57:41.446193 5896 consensus_peers.cc:485] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 -> Peer 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): Couldn't send request to peer 363f4ac4964f41a6ba0f98d1f487db83. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c59efbd84dd847188714271f4ba09ecb. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:41.446208 5295 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.446483 5829 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.446637 7970 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.446750 5694 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.446852 7969 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.447129 7969 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.447278 5426 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.447250 7970 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.447631 8324 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.448040 5563 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.449038 8324 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.451138 7970 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.452747 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet c59efbd84dd847188714271f4ba09ecb with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:41.453119 6097 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.453147 5296 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.455343 5147 catalog_manager.cc:5484] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.455603 5694 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.457684 5426 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.457813 5829 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.458379 5563 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.460467 5296 consensus_queue.cc:237] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:41.462551 5426 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.463393 5960 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.463868 8306 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.464012 8443 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.464071 6097 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.464403 5694 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.464679 5829 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.465106 8625 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.465677 8626 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.465849 5563 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:41.466375 8306 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.466675 8306 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:41.470028 8635 ts_tablet_manager.cc:902] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:41.470567 8635 tablet_copy_client.cc:308] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:41.471976 8443 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.472244 6097 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.472497 5563 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.473716 5829 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.474730 5426 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.475224 5154 catalog_manager.cc:5484] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.476845 5694 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.477100 5960 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:41.480268 5296 consensus_queue.cc:237] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.482204 5829 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.482395 5426 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.482535 8306 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.482964 6097 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.482973 5563 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.483215 5960 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.483723 8627 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.483983 8628 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.484113 8306 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.484581 5694 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:41.484958 8306 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:41.485599 8627 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240501 19:57:41.486867 5231 consensus_peers.cc:485] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 -> Peer 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): Couldn't send request to peer 363f4ac4964f41a6ba0f98d1f487db83. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85d29eb6552945a7879ee1a7e12919f0. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:41.488023 8306 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.488382 5960 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.488935 5561 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.490341 5138 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 85d29eb6552945a7879ee1a7e12919f0 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:41.490942 6097 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.491932 5426 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.492602 5694 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.492774 5153 catalog_manager.cc:5484] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.497398 5829 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:41.499446 8648 ts_tablet_manager.cc:902] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:41.499514 5716 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet 064f698b376e407da78c7d439553896a from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:49559)
I20240501 19:57:41.499579 5716 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet 064f698b376e407da78c7d439553896a from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:49559: session id = 143c54da1456472dadbf1d5d65e0a6f6-064f698b376e407da78c7d439553896a
I20240501 19:57:41.499984 8648 tablet_copy_client.cc:308] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:41.500458 5716 tablet_copy_source_session.cc:205] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:41.501233 5716 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet f27a79cf3a044383918f5d6f600a0fd9 from peer 363f4ac4964f41a6ba0f98d1f487db83 ({username='slave'} at 127.20.210.8:53017)
I20240501 19:57:41.501325 5716 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet f27a79cf3a044383918f5d6f600a0fd9 from peer 363f4ac4964f41a6ba0f98d1f487db83 at {username='slave'} at 127.20.210.8:53017: session id = 363f4ac4964f41a6ba0f98d1f487db83-f27a79cf3a044383918f5d6f600a0fd9
I20240501 19:57:41.501731 8635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064f698b376e407da78c7d439553896a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:41.502123 5716 tablet_copy_source_session.cc:205] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:41.502831 8648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f27a79cf3a044383918f5d6f600a0fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:41.503744 8635 tablet_copy_client.cc:783] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:41.503896 8635 tablet_copy_client.cc:647] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:41.504753 8648 tablet_copy_client.cc:783] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:41.504887 8648 tablet_copy_client.cc:647] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:41.505759 8635 tablet_copy_client.cc:523] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:41.506913 8635 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:41.509826 8648 tablet_copy_client.cc:523] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:41.510913 8648 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap starting.
I20240501 19:57:41.540356 8648 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:41.540696 8648 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap complete.
I20240501 19:57:41.540894 8648 ts_tablet_manager.cc:1367] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent bootstrapping tablet: real 0.030s user 0.024s sys 0.000s
I20240501 19:57:41.541296 8648 raft_consensus.cc:348] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.541426 8648 raft_consensus.cc:729] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 363f4ac4964f41a6ba0f98d1f487db83, State: Initialized, Role: LEARNER
I20240501 19:57:41.541519 8648 consensus_queue.cc:260] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.541872 8648 ts_tablet_manager.cc:1398] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:41.542622 5715 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-f27a79cf3a044383918f5d6f600a0fd9 received from {username='slave'} at 127.20.210.8:53017
I20240501 19:57:41.542706 5715 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-f27a79cf3a044383918f5d6f600a0fd9 on tablet f27a79cf3a044383918f5d6f600a0fd9 with peer 363f4ac4964f41a6ba0f98d1f487db83
I20240501 19:57:41.544044 8635 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:41.544309 8635 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:41.544464 8635 ts_tablet_manager.cc:1367] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.038s user 0.014s sys 0.009s
I20240501 19:57:41.544848 8635 raft_consensus.cc:348] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:41.544986 8635 raft_consensus.cc:729] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:41.545086 8635 consensus_queue.cc:260] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:41.546572 8635 ts_tablet_manager.cc:1398] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240501 19:57:41.547626 5715 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-064f698b376e407da78c7d439553896a received from {username='slave'} at 127.20.210.9:49559
I20240501 19:57:41.547693 5715 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-064f698b376e407da78c7d439553896a on tablet 064f698b376e407da78c7d439553896a with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:41.576488 8648 ts_tablet_manager.cc:902] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83: Initiating tablet copy from peer 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085)
I20240501 19:57:41.576865 8648 tablet_copy_client.cc:308] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.6:34085
I20240501 19:57:41.579843 5983 tablet_copy_service.cc:140] P 9544b0e566c54a62a6135f3bff47c4f2: Received BeginTabletCopySession request for tablet 2f89a8073ad64580b7aa8612204b3d38 from peer 363f4ac4964f41a6ba0f98d1f487db83 ({username='slave'} at 127.20.210.8:35853)
I20240501 19:57:41.579923 5983 tablet_copy_service.cc:161] P 9544b0e566c54a62a6135f3bff47c4f2: Beginning new tablet copy session on tablet 2f89a8073ad64580b7aa8612204b3d38 from peer 363f4ac4964f41a6ba0f98d1f487db83 at {username='slave'} at 127.20.210.8:35853: session id = 363f4ac4964f41a6ba0f98d1f487db83-2f89a8073ad64580b7aa8612204b3d38
I20240501 19:57:41.580669 5983 tablet_copy_source_session.cc:205] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:41.581522 8648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f89a8073ad64580b7aa8612204b3d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:41.583254 8648 tablet_copy_client.cc:783] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:41.583407 8648 tablet_copy_client.cc:647] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:41.584687 8648 tablet_copy_client.cc:523] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:41.585558 8648 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap starting.
I20240501 19:57:41.603561 8648 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:41.603821 8648 tablet_bootstrap.cc:492] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap complete.
I20240501 19:57:41.603971 8648 ts_tablet_manager.cc:1367] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.004s
I20240501 19:57:41.604290 8648 raft_consensus.cc:348] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.604388 8648 raft_consensus.cc:729] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 363f4ac4964f41a6ba0f98d1f487db83, State: Initialized, Role: LEARNER
I20240501 19:57:41.604478 8648 consensus_queue.cc:260] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.604710 8648 ts_tablet_manager.cc:1398] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:41.605113 5983 tablet_copy_service.cc:342] P 9544b0e566c54a62a6135f3bff47c4f2: Request end of tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-2f89a8073ad64580b7aa8612204b3d38 received from {username='slave'} at 127.20.210.8:35853
I20240501 19:57:41.605186 5983 tablet_copy_service.cc:433] P 9544b0e566c54a62a6135f3bff47c4f2: ending tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-2f89a8073ad64580b7aa8612204b3d38 on tablet 2f89a8073ad64580b7aa8612204b3d38 with peer 363f4ac4964f41a6ba0f98d1f487db83
I20240501 19:57:41.763273 8648 ts_tablet_manager.cc:902] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:41.763808 8648 tablet_copy_client.cc:308] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:41.768041 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet c6ba23a7b4d24e0aa077e47798e890b8 from peer 363f4ac4964f41a6ba0f98d1f487db83 ({username='slave'} at 127.20.210.8:48011)
I20240501 19:57:41.768164 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet c6ba23a7b4d24e0aa077e47798e890b8 from peer 363f4ac4964f41a6ba0f98d1f487db83 at {username='slave'} at 127.20.210.8:48011: session id = 363f4ac4964f41a6ba0f98d1f487db83-c6ba23a7b4d24e0aa077e47798e890b8
I20240501 19:57:41.769325 5583 tablet_copy_source_session.cc:205] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:41.770318 8648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ba23a7b4d24e0aa077e47798e890b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:41.773031 8648 tablet_copy_client.cc:783] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:41.773185 8648 tablet_copy_client.cc:647] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:41.774663 8648 tablet_copy_client.cc:523] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:41.776321 8648 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap starting.
I20240501 19:57:41.799243 8635 ts_tablet_manager.cc:902] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:41.799638 8635 tablet_copy_client.cc:308] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:41.804823 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet 80fdea49202c4ab5b3972b2c9ad7cff5 from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:42563)
I20240501 19:57:41.804925 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet 80fdea49202c4ab5b3972b2c9ad7cff5 from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:42563: session id = 143c54da1456472dadbf1d5d65e0a6f6-80fdea49202c4ab5b3972b2c9ad7cff5
I20240501 19:57:41.805840 5583 tablet_copy_source_session.cc:205] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:41.806671 8635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80fdea49202c4ab5b3972b2c9ad7cff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:41.809154 8635 tablet_copy_client.cc:783] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:41.809321 8635 tablet_copy_client.cc:647] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:41.810355 8635 tablet_copy_client.cc:523] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:41.811329 8635 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:41.838174 8635 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:41.838439 8635 tablet_bootstrap.cc:492] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:41.838605 8635 ts_tablet_manager.cc:1367] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.027s user 0.018s sys 0.003s
I20240501 19:57:41.839069 8635 raft_consensus.cc:348] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:41.839452 8635 raft_consensus.cc:729] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:41.839565 8635 consensus_queue.cc:260] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:41.841526 8635 ts_tablet_manager.cc:1398] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240501 19:57:41.842092 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-80fdea49202c4ab5b3972b2c9ad7cff5 received from {username='slave'} at 127.20.210.9:42563
I20240501 19:57:41.842155 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-80fdea49202c4ab5b3972b2c9ad7cff5 on tablet 80fdea49202c4ab5b3972b2c9ad7cff5 with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:41.848057 8744 ts_tablet_manager.cc:902] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83: Initiating tablet copy from peer 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085)
I20240501 19:57:41.848657 8744 tablet_copy_client.cc:308] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.6:34085
I20240501 19:57:41.849161 5983 tablet_copy_service.cc:140] P 9544b0e566c54a62a6135f3bff47c4f2: Received BeginTabletCopySession request for tablet c59efbd84dd847188714271f4ba09ecb from peer 363f4ac4964f41a6ba0f98d1f487db83 ({username='slave'} at 127.20.210.8:35853)
I20240501 19:57:41.849265 5983 tablet_copy_service.cc:161] P 9544b0e566c54a62a6135f3bff47c4f2: Beginning new tablet copy session on tablet c59efbd84dd847188714271f4ba09ecb from peer 363f4ac4964f41a6ba0f98d1f487db83 at {username='slave'} at 127.20.210.8:35853: session id = 363f4ac4964f41a6ba0f98d1f487db83-c59efbd84dd847188714271f4ba09ecb
I20240501 19:57:41.850458 5983 tablet_copy_source_session.cc:205] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:41.853319 8648 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:41.853623 8648 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap complete.
I20240501 19:57:41.853786 8648 ts_tablet_manager.cc:1367] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent bootstrapping tablet: real 0.078s user 0.039s sys 0.000s
I20240501 19:57:41.854136 8648 raft_consensus.cc:348] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.855063 8648 raft_consensus.cc:729] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 363f4ac4964f41a6ba0f98d1f487db83, State: Initialized, Role: LEARNER
I20240501 19:57:41.854578 8744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c59efbd84dd847188714271f4ba09ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:41.855273 8648 consensus_queue.cc:260] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.857100 8648 ts_tablet_manager.cc:1398] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20240501 19:57:41.857616 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-c6ba23a7b4d24e0aa077e47798e890b8 received from {username='slave'} at 127.20.210.8:48011
I20240501 19:57:41.857689 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-c6ba23a7b4d24e0aa077e47798e890b8 on tablet c6ba23a7b4d24e0aa077e47798e890b8 with peer 363f4ac4964f41a6ba0f98d1f487db83
I20240501 19:57:41.858147 8744 tablet_copy_client.cc:783] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:41.858325 8744 tablet_copy_client.cc:647] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:41.860105 8744 tablet_copy_client.cc:523] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:41.861726 8744 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap starting.
I20240501 19:57:41.865401 8648 ts_tablet_manager.cc:902] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83: Initiating tablet copy from peer 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499)
I20240501 19:57:41.865866 8648 tablet_copy_client.cc:308] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.5:39499
I20240501 19:57:41.866415 5849 tablet_copy_service.cc:140] P 1865147cd5a346ad8549f18fa8cc26c3: Received BeginTabletCopySession request for tablet e17a6ae467bf42abac714a94e1ec28e5 from peer 363f4ac4964f41a6ba0f98d1f487db83 ({username='slave'} at 127.20.210.8:59559)
I20240501 19:57:41.866504 5849 tablet_copy_service.cc:161] P 1865147cd5a346ad8549f18fa8cc26c3: Beginning new tablet copy session on tablet e17a6ae467bf42abac714a94e1ec28e5 from peer 363f4ac4964f41a6ba0f98d1f487db83 at {username='slave'} at 127.20.210.8:59559: session id = 363f4ac4964f41a6ba0f98d1f487db83-e17a6ae467bf42abac714a94e1ec28e5
I20240501 19:57:41.867302 5849 tablet_copy_source_session.cc:205] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:41.867976 8648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e17a6ae467bf42abac714a94e1ec28e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:41.869822 8648 tablet_copy_client.cc:783] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:41.869954 8648 tablet_copy_client.cc:647] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:41.874245 8648 tablet_copy_client.cc:523] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:41.875581 8648 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap starting.
I20240501 19:57:41.897648 8648 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:41.897862 8648 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap complete.
I20240501 19:57:41.898001 8648 ts_tablet_manager.cc:1367] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent bootstrapping tablet: real 0.022s user 0.014s sys 0.000s
I20240501 19:57:41.898542 8648 raft_consensus.cc:348] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.898640 8648 raft_consensus.cc:729] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 363f4ac4964f41a6ba0f98d1f487db83, State: Initialized, Role: LEARNER
I20240501 19:57:41.898746 8648 consensus_queue.cc:260] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.899080 8648 ts_tablet_manager.cc:1398] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent starting tablet: real 0.001s user 0.005s sys 0.000s
I20240501 19:57:41.899554 5849 tablet_copy_service.cc:342] P 1865147cd5a346ad8549f18fa8cc26c3: Request end of tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-e17a6ae467bf42abac714a94e1ec28e5 received from {username='slave'} at 127.20.210.8:59559
I20240501 19:57:41.899621 5849 tablet_copy_service.cc:433] P 1865147cd5a346ad8549f18fa8cc26c3: ending tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-e17a6ae467bf42abac714a94e1ec28e5 on tablet e17a6ae467bf42abac714a94e1ec28e5 with peer 363f4ac4964f41a6ba0f98d1f487db83
I20240501 19:57:41.899979 8744 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:41.900238 8744 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap complete.
I20240501 19:57:41.900380 8744 ts_tablet_manager.cc:1367] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83: Time spent bootstrapping tablet: real 0.039s user 0.032s sys 0.002s
I20240501 19:57:41.900903 8744 raft_consensus.cc:348] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.901028 8744 raft_consensus.cc:729] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 363f4ac4964f41a6ba0f98d1f487db83, State: Initialized, Role: LEARNER
I20240501 19:57:41.901134 8744 consensus_queue.cc:260] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.901412 8744 ts_tablet_manager.cc:1398] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:41.901896 5983 tablet_copy_service.cc:342] P 9544b0e566c54a62a6135f3bff47c4f2: Request end of tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-c59efbd84dd847188714271f4ba09ecb received from {username='slave'} at 127.20.210.8:35853
I20240501 19:57:41.901974 5983 tablet_copy_service.cc:433] P 9544b0e566c54a62a6135f3bff47c4f2: ending tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-c59efbd84dd847188714271f4ba09ecb on tablet c59efbd84dd847188714271f4ba09ecb with peer 363f4ac4964f41a6ba0f98d1f487db83
I20240501 19:57:41.922277 8744 ts_tablet_manager.cc:902] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83: Initiating tablet copy from peer ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285)
I20240501 19:57:41.922642 8744 tablet_copy_client.cc:308] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:46285
I20240501 19:57:41.925259 5317 tablet_copy_service.cc:140] P ad52038f1d38489ea464a8fbc72daac0: Received BeginTabletCopySession request for tablet 85d29eb6552945a7879ee1a7e12919f0 from peer 363f4ac4964f41a6ba0f98d1f487db83 ({username='slave'} at 127.20.210.8:58425)
I20240501 19:57:41.925339 5317 tablet_copy_service.cc:161] P ad52038f1d38489ea464a8fbc72daac0: Beginning new tablet copy session on tablet 85d29eb6552945a7879ee1a7e12919f0 from peer 363f4ac4964f41a6ba0f98d1f487db83 at {username='slave'} at 127.20.210.8:58425: session id = 363f4ac4964f41a6ba0f98d1f487db83-85d29eb6552945a7879ee1a7e12919f0
I20240501 19:57:41.925935 5317 tablet_copy_source_session.cc:205] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:41.926584 8744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d29eb6552945a7879ee1a7e12919f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:41.928335 8744 tablet_copy_client.cc:783] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:41.928615 8744 tablet_copy_client.cc:647] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:41.928752 7550 raft_consensus.cc:1203] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:41.930585 8744 tablet_copy_client.cc:523] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:41.931669 8744 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap starting.
I20240501 19:57:41.940526 8635 ts_tablet_manager.cc:902] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499)
I20240501 19:57:41.940943 8635 tablet_copy_client.cc:308] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.5:39499
I20240501 19:57:41.944998 5849 tablet_copy_service.cc:140] P 1865147cd5a346ad8549f18fa8cc26c3: Received BeginTabletCopySession request for tablet 4a713bc0643d4bdf91f669dde94f63f8 from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:33281)
I20240501 19:57:41.945088 5849 tablet_copy_service.cc:161] P 1865147cd5a346ad8549f18fa8cc26c3: Beginning new tablet copy session on tablet 4a713bc0643d4bdf91f669dde94f63f8 from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:33281: session id = 143c54da1456472dadbf1d5d65e0a6f6-4a713bc0643d4bdf91f669dde94f63f8
I20240501 19:57:41.945982 5849 tablet_copy_source_session.cc:205] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:41.946751 8635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a713bc0643d4bdf91f669dde94f63f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:41.948542 8635 tablet_copy_client.cc:783] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:41.948660 8635 tablet_copy_client.cc:647] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:41.950498 8635 tablet_copy_client.cc:523] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:41.951506 8635 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:41.952095 8744 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:41.952320 8744 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap complete.
I20240501 19:57:41.952466 8744 ts_tablet_manager.cc:1367] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent bootstrapping tablet: real 0.021s user 0.014s sys 0.004s
I20240501 19:57:41.952833 8744 raft_consensus.cc:348] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.952970 8744 raft_consensus.cc:729] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 363f4ac4964f41a6ba0f98d1f487db83, State: Initialized, Role: LEARNER
I20240501 19:57:41.953073 8744 consensus_queue.cc:260] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:41.953423 8744 ts_tablet_manager.cc:1398] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 19:57:41.953796 5317 tablet_copy_service.cc:342] P ad52038f1d38489ea464a8fbc72daac0: Request end of tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-85d29eb6552945a7879ee1a7e12919f0 received from {username='slave'} at 127.20.210.8:58425
I20240501 19:57:41.953861 5317 tablet_copy_service.cc:433] P ad52038f1d38489ea464a8fbc72daac0: ending tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-85d29eb6552945a7879ee1a7e12919f0 on tablet 85d29eb6552945a7879ee1a7e12919f0 with peer 363f4ac4964f41a6ba0f98d1f487db83
I20240501 19:57:41.969406 8635 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:41.969622 8635 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:41.969776 8635 ts_tablet_manager.cc:1367] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.001s
I20240501 19:57:41.970070 8635 raft_consensus.cc:348] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:41.970150 8635 raft_consensus.cc:729] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:41.970212 8635 consensus_queue.cc:260] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:41.970471 8635 ts_tablet_manager.cc:1398] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 19:57:41.970927 5849 tablet_copy_service.cc:342] P 1865147cd5a346ad8549f18fa8cc26c3: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-4a713bc0643d4bdf91f669dde94f63f8 received from {username='slave'} at 127.20.210.9:33281
I20240501 19:57:41.970997 5849 tablet_copy_service.cc:433] P 1865147cd5a346ad8549f18fa8cc26c3: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-4a713bc0643d4bdf91f669dde94f63f8 on tablet 4a713bc0643d4bdf91f669dde94f63f8 with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:42.064522 7418 raft_consensus.cc:1203] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:42.069387 7418 raft_consensus.cc:1203] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:42.253785 7418 raft_consensus.cc:1203] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:42.326962 7418 raft_consensus.cc:1203] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:42.329895 7418 raft_consensus.cc:1203] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:42.400976 7550 raft_consensus.cc:1203] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:42.406347 8417 raft_consensus.cc:1050] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:42.406870 8417 consensus_queue.cc:237] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:42.408689 5295 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.408809 6097 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.408934 5561 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.409245 8417 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.409333 7550 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.409441 5827 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.409550 8417 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.409822 8417 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.409934 5960 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.409932 8880 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.410270 8417 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.411141 8417 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.411257 5429 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.412313 8879 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.413030 7253 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.413646 7550 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.415102 6097 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.415930 5147 catalog_manager.cc:5484] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 reported cstate change: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.418010 5827 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.418022 5960 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.421332 5429 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.422077 5295 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.424963 5561 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.428831 5696 consensus_queue.cc:237] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:42.430166 6097 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.430265 5429 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.430269 5561 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.430276 7550 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.430639 8891 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.430735 5295 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.430846 8415 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.430999 8892 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.431147 7253 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.431228 5960 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.431407 8878 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.431954 8892 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.433771 8892 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.434165 7550 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.434716 5960 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.434785 5295 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.436465 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 064f698b376e407da78c7d439553896a with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:42.436458 5145 catalog_manager.cc:5484] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:42.437528 5561 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.437563 6097 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.438876 5429 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.440209 5809 tablet_service.cc:1514] Processing DeleteTablet for tablet 064f698b376e407da78c7d439553896a with delete_type TABLET_DATA_TOMBSTONED (TS 1865147cd5a346ad8549f18fa8cc26c3 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:35230
I20240501 19:57:42.442601 8895 tablet_replica.cc:330] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3: stopping tablet replica
I20240501 19:57:42.442785 8895 raft_consensus.cc:2229] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:42.443092 8895 raft_consensus.cc:2258] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:42.443814 7418 raft_consensus.cc:1203] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:42.445341 8895 ts_tablet_manager.cc:1875] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:42.447073 8895 ts_tablet_manager.cc:1888] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:42.447149 8895 log.cc:1196] T 064f698b376e407da78c7d439553896a P 1865147cd5a346ad8549f18fa8cc26c3: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/wals/064f698b376e407da78c7d439553896a
I20240501 19:57:42.447588 5139 catalog_manager.cc:4830] TS 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499): tablet 064f698b376e407da78c7d439553896a (table rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]) successfully deleted
I20240501 19:57:42.480170 8782 raft_consensus.cc:1050] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2: attempting to promote NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 to VOTER
I20240501 19:57:42.480233 7550 raft_consensus.cc:1203] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:42.480719 8782 consensus_queue.cc:237] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:42.482429 7418 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.482771 5827 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.482914 5696 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.483017 8897 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.483110 5429 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.483177 8782 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.483322 8781 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.483453 5295 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.483793 6097 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.484344 8782 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.482818 5561 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.484825 8782 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.484957 8897 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.485054 8781 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.487110 8781 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.488109 5696 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.488108 7418 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.489136 5295 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.489794 5561 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.490181 5827 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.490531 5152 catalog_manager.cc:5484] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83 reported cstate change: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.491979 5429 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.494853 6097 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.500133 5962 consensus_queue.cc:237] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:42.501760 7418 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.501739 5827 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.502020 6097 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.502020 5295 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.502312 8782 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.502393 5429 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.502411 5561 raft_consensus.cc:1261] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.502616 8782 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.502765 8897 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.502893 8899 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.503005 7969 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.503296 8782 consensus_queue.cc:1035] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.505339 8781 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.505863 5295 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.505888 5429 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.506466 6097 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.507545 5827 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.508443 5154 catalog_manager.cc:5484] T 2f89a8073ad64580b7aa8612204b3d38 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.508860 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2f89a8073ad64580b7aa8612204b3d38 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:42.509811 5561 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.510843 5674 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f89a8073ad64580b7aa8612204b3d38 with delete_type TABLET_DATA_TOMBSTONED (TS 4d4855c096ac4e1e936abf15834332ae not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:36532
I20240501 19:57:42.511183 8909 tablet_replica.cc:330] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae: stopping tablet replica
I20240501 19:57:42.511576 8909 raft_consensus.cc:2229] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:42.511771 8909 raft_consensus.cc:2258] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:42.512502 8909 ts_tablet_manager.cc:1875] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:42.512389 7418 raft_consensus.cc:2932] T 2f89a8073ad64580b7aa8612204b3d38 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.514559 8909 ts_tablet_manager.cc:1888] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:42.514678 8909 log.cc:1196] T 2f89a8073ad64580b7aa8612204b3d38 P 4d4855c096ac4e1e936abf15834332ae: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/2f89a8073ad64580b7aa8612204b3d38
I20240501 19:57:42.515868 5141 catalog_manager.cc:4830] TS 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519): tablet 2f89a8073ad64580b7aa8612204b3d38 (table rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]) successfully deleted
I20240501 19:57:42.532600 8879 raft_consensus.cc:1050] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 to VOTER
I20240501 19:57:42.533033 8879 consensus_queue.cc:237] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:42.534657 5827 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.534993 5561 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.534993 5429 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.535060 5962 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.535302 5295 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.535353 8878 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.535588 7418 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.535609 6097 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.535746 8892 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.536029 8880 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.536229 8878 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.536368 8880 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.536620 8415 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.538581 8415 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.540041 8892 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.540424 5295 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.540879 7418 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.541379 5962 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.542647 5429 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.543821 5150 catalog_manager.cc:5484] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.543848 6097 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.546794 5827 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.550544 5561 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.552306 5694 consensus_queue.cc:237] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:42.553894 6097 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.553934 5429 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.554188 8880 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.554318 8879 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.554801 5561 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.555220 8880 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.555444 5827 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.555948 8880 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.555975 7418 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.556380 8880 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.557471 8880 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.558054 7418 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.558073 5827 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.558476 5295 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.559551 5429 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.559600 6097 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.560199 5150 catalog_manager.cc:5484] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 reported cstate change: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.561094 8880 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.561604 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f27a79cf3a044383918f5d6f600a0fd9 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:42.562009 5561 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.562213 5295 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.563210 5942 tablet_service.cc:1514] Processing DeleteTablet for tablet f27a79cf3a044383918f5d6f600a0fd9 with delete_type TABLET_DATA_TOMBSTONED (TS 9544b0e566c54a62a6135f3bff47c4f2 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:42138
I20240501 19:57:42.565078 8918 tablet_replica.cc:330] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2: stopping tablet replica
I20240501 19:57:42.565280 8918 raft_consensus.cc:2229] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:42.565517 8918 raft_consensus.cc:2258] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:42.566182 8918 ts_tablet_manager.cc:1875] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:42.567963 8918 ts_tablet_manager.cc:1888] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:42.568058 8918 log.cc:1196] T f27a79cf3a044383918f5d6f600a0fd9 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/wals/f27a79cf3a044383918f5d6f600a0fd9
I20240501 19:57:42.568543 5141 catalog_manager.cc:4830] TS 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085): tablet f27a79cf3a044383918f5d6f600a0fd9 (table rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]) successfully deleted
I20240501 19:57:42.674391 8573 raft_consensus.cc:1050] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 to VOTER
I20240501 19:57:42.674803 8573 consensus_queue.cc:237] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:42.676391 7418 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.676658 5295 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.677117 8969 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.677296 5429 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.677489 8969 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.677613 5827 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.678313 5962 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.678409 5694 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.678361 8966 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.678727 8967 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.678759 6097 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.678871 8970 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.679193 8970 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.679852 8970 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.682088 8970 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.682607 5694 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.682636 6097 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.682636 5827 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.684998 5295 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.685663 5962 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.685791 5150 catalog_manager.cc:5484] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:42.685868 5429 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.688140 7418 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.692849 5561 consensus_queue.cc:237] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:42.694272 5962 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.694309 5295 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.694309 5429 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.694319 5694 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.694633 6097 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.694653 8829 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.694695 7418 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.694783 8967 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.694919 8573 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.695036 8343 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.695127 8969 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.696756 8966 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.697160 8343 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.697592 5962 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.698072 6097 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.698534 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c6ba23a7b4d24e0aa077e47798e890b8 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:42.698937 5429 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.699811 5295 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.700919 5694 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.702067 5149 catalog_manager.cc:5484] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.704890 5809 tablet_service.cc:1514] Processing DeleteTablet for tablet c6ba23a7b4d24e0aa077e47798e890b8 with delete_type TABLET_DATA_TOMBSTONED (TS 1865147cd5a346ad8549f18fa8cc26c3 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:35230
I20240501 19:57:42.705138 8895 tablet_replica.cc:330] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3: stopping tablet replica
I20240501 19:57:42.705282 8895 raft_consensus.cc:2229] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:42.705456 8895 raft_consensus.cc:2258] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:42.706096 8895 ts_tablet_manager.cc:1875] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:42.707134 7418 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.708038 8895 ts_tablet_manager.cc:1888] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:42.708109 8895 log.cc:1196] T c6ba23a7b4d24e0aa077e47798e890b8 P 1865147cd5a346ad8549f18fa8cc26c3: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/wals/c6ba23a7b4d24e0aa077e47798e890b8
I20240501 19:57:42.710086 5139 catalog_manager.cc:4830] TS 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499): tablet c6ba23a7b4d24e0aa077e47798e890b8 (table rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]) successfully deleted
I20240501 19:57:42.752863 8907 raft_consensus.cc:1050] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2: attempting to promote NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 to VOTER
I20240501 19:57:42.753311 8907 consensus_queue.cc:237] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:42.754860 5561 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.754920 5296 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.755290 6097 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.755288 8781 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.755616 8781 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.755920 8781 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.756884 5429 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.757005 5827 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.757005 7418 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.757265 8781 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.757382 8899 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.757623 5694 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.757679 8899 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.757992 8899 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.760274 8899 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.760704 5429 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.761070 6097 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.762946 5296 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.763551 5147 catalog_manager.cc:5484] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.765168 5827 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.767437 5561 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.767830 7418 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.771317 5694 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.772447 5962 consensus_queue.cc:237] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:42.773801 5561 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.774036 7418 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.774112 8897 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.774317 5429 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.774562 5827 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.774804 6097 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.775141 7969 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.775236 8781 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.775432 7969 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.775588 8907 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.777854 8897 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.778402 5296 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.778784 5429 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.779510 6095 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.779915 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c59efbd84dd847188714271f4ba09ecb with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:42.780086 8908 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.780262 5827 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.780922 5561 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.781643 7418 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.783056 5149 catalog_manager.cc:5484] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:42.784539 5296 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.786255 5674 tablet_service.cc:1514] Processing DeleteTablet for tablet c59efbd84dd847188714271f4ba09ecb with delete_type TABLET_DATA_TOMBSTONED (TS 4d4855c096ac4e1e936abf15834332ae not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:36532
I20240501 19:57:42.786461 8909 tablet_replica.cc:330] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae: stopping tablet replica
I20240501 19:57:42.786582 8909 raft_consensus.cc:2229] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:42.786723 8909 raft_consensus.cc:2258] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:42.787566 8909 ts_tablet_manager.cc:1875] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:42.789276 8909 ts_tablet_manager.cc:1888] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:42.789335 8909 log.cc:1196] T c59efbd84dd847188714271f4ba09ecb P 4d4855c096ac4e1e936abf15834332ae: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/c59efbd84dd847188714271f4ba09ecb
I20240501 19:57:42.789784 5141 catalog_manager.cc:4830] TS 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519): tablet c59efbd84dd847188714271f4ba09ecb (table rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]) successfully deleted
I20240501 19:57:42.804037 8968 raft_consensus.cc:1050] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:42.804507 8968 consensus_queue.cc:237] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:42.806182 7550 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.806228 6095 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.806363 5296 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.806577 5962 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.806721 8968 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.806887 8970 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.806983 5694 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.806983 5429 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.807327 8970 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.807514 8966 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.807619 8967 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.808594 5827 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.809351 8573 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.809566 8966 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.810686 8573 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.811504 5296 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.811834 7550 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.813154 5694 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.813136 6095 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.814456 5147 catalog_manager.cc:5484] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6 reported cstate change: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.814941 5429 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.816838 5962 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.817327 5827 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.826428 5561 consensus_queue.cc:237] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:42.827900 7550 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.827900 5429 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.828320 6095 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.828336 5827 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.828384 5962 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.828639 8970 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.828770 8967 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.828850 8968 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.829419 8970 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.829689 8968 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.830235 5296 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.830583 8967 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.832123 8970 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.832504 5296 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.833652 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 80fdea49202c4ab5b3972b2c9ad7cff5 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:42.833902 6095 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.834070 5827 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.834453 5145 catalog_manager.cc:5484] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:42.836114 5962 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.836374 7550 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.837687 5674 tablet_service.cc:1514] Processing DeleteTablet for tablet 80fdea49202c4ab5b3972b2c9ad7cff5 with delete_type TABLET_DATA_TOMBSTONED (TS 4d4855c096ac4e1e936abf15834332ae not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:36532
I20240501 19:57:42.837869 8909 tablet_replica.cc:330] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae: stopping tablet replica
I20240501 19:57:42.837996 8909 raft_consensus.cc:2229] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:42.838166 8909 raft_consensus.cc:2258] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:42.838799 8909 ts_tablet_manager.cc:1875] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:42.840687 8909 ts_tablet_manager.cc:1888] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:42.840754 8909 log.cc:1196] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 4d4855c096ac4e1e936abf15834332ae: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/80fdea49202c4ab5b3972b2c9ad7cff5
I20240501 19:57:42.841027 5429 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:42.842377 5141 catalog_manager.cc:4830] TS 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519): tablet 80fdea49202c4ab5b3972b2c9ad7cff5 (table rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]) successfully deleted
I20240501 19:57:42.845820 5694 consensus_queue.cc:237] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:42.847280 6095 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:42.847412 5295 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:42.847417 5429 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:42.847506 5561 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:42.847749 8892 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:42.847868 5827 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:42.847894 8415 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:42.847988 8417 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:42.848173 8880 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:42.848290 8417 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:42.849124 5962 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:42.849474 8415 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:42.851256 8879 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:42.851858 5829 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:42.851925 5429 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:42.852166 6095 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:42.854575 5149 catalog_manager.cc:5484] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:42.854856 5561 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:42.856475 5962 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:42.857497 5694 consensus_queue.cc:237] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:42.859447 5429 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240501 19:57:42.859563 5629 consensus_peers.cc:485] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cb5540ce8d3343a68a8bf7d8fff00589. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:42.859651 5296 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:42.859902 8417 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:42.859794 5295 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:42.860419 6095 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:42.860602 5561 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:42.860658 5962 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:42.860864 8891 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:42.861101 8415 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:42.861364 8415 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:42.861660 8415 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:42.863508 5829 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:42.863955 8880 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:42.864315 8415 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:42.864634 5429 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:42.865094 5829 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:42.866014 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet cb5540ce8d3343a68a8bf7d8fff00589 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:42.866564 6095 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:42.866637 5962 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:42.866950 5145 catalog_manager.cc:5484] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:42.869143 5561 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:42.870766 5295 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:42.883054 8830 raft_consensus.cc:1050] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0: attempting to promote NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 to VOTER
I20240501 19:57:42.883476 8830 consensus_queue.cc:237] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:42.885353 5962 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.885654 5561 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.885829 8830 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.886093 7418 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.886487 6095 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.886567 5429 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.886641 8646 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.886817 5829 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.886914 8830 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.887243 8830 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.887495 8828 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.888900 5694 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:42.889318 9022 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.890369 8830 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:42.891372 9022 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.891824 5694 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.893193 6095 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.894089 5429 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.894337 5829 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.896667 5154 catalog_manager.cc:5484] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:42.897650 5962 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.898123 7418 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.903301 5561 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.907078 5295 consensus_queue.cc:237] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:42.908710 5962 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.908807 6095 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.909250 7418 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.909245 9023 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.909483 5694 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.909577 9022 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.909709 9023 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.909914 8828 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.910638 5429 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.911023 8828 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.911883 5561 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:42.912806 5694 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.913055 7418 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.913054 8828 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.914311 8830 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:42.914530 5429 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.915052 5138 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 85d29eb6552945a7879ee1a7e12919f0 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:42.915614 5145 catalog_manager.cc:5484] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.916172 6097 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.916440 5962 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.917873 5561 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:42.921763 5809 tablet_service.cc:1514] Processing DeleteTablet for tablet 85d29eb6552945a7879ee1a7e12919f0 with delete_type TABLET_DATA_TOMBSTONED (TS 1865147cd5a346ad8549f18fa8cc26c3 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:35230
I20240501 19:57:42.922163 8895 tablet_replica.cc:330] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3: stopping tablet replica
I20240501 19:57:42.922315 8895 raft_consensus.cc:2229] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:42.922487 8895 raft_consensus.cc:2258] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:42.923221 8895 ts_tablet_manager.cc:1875] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:42.925046 8895 ts_tablet_manager.cc:1888] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:42.925108 8895 log.cc:1196] T 85d29eb6552945a7879ee1a7e12919f0 P 1865147cd5a346ad8549f18fa8cc26c3: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/wals/85d29eb6552945a7879ee1a7e12919f0
I20240501 19:57:42.925653 5139 catalog_manager.cc:4830] TS 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499): tablet 85d29eb6552945a7879ee1a7e12919f0 (table rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]) successfully deleted
I20240501 19:57:43.002523 8554 raft_consensus.cc:1050] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:43.002979 8554 consensus_queue.cc:237] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:43.004712 6097 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.004739 5429 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.004778 5962 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.005120 5694 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.005100 9056 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.005218 7550 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.005347 8554 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.005486 9057 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.005668 5295 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.005889 8554 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.005983 5561 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.006290 8359 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.006551 8359 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.007279 9057 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.009013 8359 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.009321 5429 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.009733 5561 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.010808 7550 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.012259 5150 catalog_manager.cc:5484] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:43.013134 5295 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.013473 5962 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.013800 5694 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.014644 6097 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.018952 5829 consensus_queue.cc:237] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:43.020390 6097 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:43.020395 7550 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:43.020429 5694 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:43.020503 5295 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:43.020802 5429 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:43.020828 5561 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:43.020730 9055 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:43.021018 8359 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:43.021188 8554 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:43.021319 8359 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:43.021441 9057 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:43.022212 9055 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:43.023597 8554 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.024029 5694 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.024245 6097 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.025128 5139 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4a713bc0643d4bdf91f669dde94f63f8 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:43.026579 5563 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.026625 5150 catalog_manager.cc:5484] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.026710 7550 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.026837 5429 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.028951 5295 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:43.030778 5942 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a713bc0643d4bdf91f669dde94f63f8 with delete_type TABLET_DATA_TOMBSTONED (TS 9544b0e566c54a62a6135f3bff47c4f2 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:42138
I20240501 19:57:43.031047 8918 tablet_replica.cc:330] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2: stopping tablet replica
I20240501 19:57:43.031208 8918 raft_consensus.cc:2229] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:43.031391 8918 raft_consensus.cc:2258] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:43.032500 8918 ts_tablet_manager.cc:1875] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:43.034453 8918 ts_tablet_manager.cc:1888] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:43.034521 8918 log.cc:1196] T 4a713bc0643d4bdf91f669dde94f63f8 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/wals/4a713bc0643d4bdf91f669dde94f63f8
I20240501 19:57:43.035122 5141 catalog_manager.cc:4830] TS 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085): tablet 4a713bc0643d4bdf91f669dde94f63f8 (table rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]) successfully deleted
I20240501 19:57:43.053123 9055 raft_consensus.cc:1050] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3: attempting to promote NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 to VOTER
I20240501 19:57:43.053454 9055 consensus_queue.cc:237] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:43.054905 5561 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.054905 5295 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.054913 7418 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.055202 9057 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.055357 8359 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.055495 5694 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.055577 9055 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.055847 8554 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.056012 5429 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.056257 5962 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.056674 6097 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:43.057108 8359 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.057245 9054 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.057556 9054 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:43.059249 9054 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.059676 5962 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.060271 6097 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.061107 5295 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.061930 5429 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.062824 5694 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.062832 7418 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.063385 5561 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.065001 5154 catalog_manager.cc:5484] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 5 to 6, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:43.068900 5829 consensus_queue.cc:237] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:43.070331 7418 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:43.070541 5295 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:43.070895 9067 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:43.071161 9057 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:43.071604 6097 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:43.072053 5561 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:43.072088 9067 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:43.072355 9067 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:43.072963 5429 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:43.072963 5694 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:43.073439 9057 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:43.073575 9067 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:43.074185 9056 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.075605 5429 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.075979 5139 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e17a6ae467bf42abac714a94e1ec28e5 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:43.076149 7418 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.076591 5152 catalog_manager.cc:5484] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:43.077610 6097 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.079020 5942 tablet_service.cc:1514] Processing DeleteTablet for tablet e17a6ae467bf42abac714a94e1ec28e5 with delete_type TABLET_DATA_TOMBSTONED (TS 9544b0e566c54a62a6135f3bff47c4f2 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:42138
I20240501 19:57:43.079195 8918 tablet_replica.cc:330] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2: stopping tablet replica
I20240501 19:57:43.079319 8918 raft_consensus.cc:2229] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:43.079282 5694 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.079469 8918 raft_consensus.cc:2258] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:43.080210 8918 ts_tablet_manager.cc:1875] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:43.081405 5295 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.082211 8918 ts_tablet_manager.cc:1888] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:43.082288 8918 log.cc:1196] T e17a6ae467bf42abac714a94e1ec28e5 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/wals/e17a6ae467bf42abac714a94e1ec28e5
I20240501 19:57:43.082837 5141 catalog_manager.cc:4830] TS 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085): tablet e17a6ae467bf42abac714a94e1ec28e5 (table rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]) successfully deleted
I20240501 19:57:43.083060 5561 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:43.135857 5429 consensus_queue.cc:237] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } }
I20240501 19:57:43.137485 5962 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:43.137521 5295 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:43.137534 5694 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:43.137617 5829 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:43.137892 6097 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:43.137863 9082 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:43.138034 9047 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:43.138183 5561 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:43.138504 6818 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:43.138630 9082 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:43.138725 9084 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:43.139062 9083 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:43.141172 9083 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:43.141583 5561 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:43.142076 6097 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:43.143841 5962 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:43.144030 5145 catalog_manager.cc:5484] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "9782508a47ba4403a39ca49fff7677fa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } }
I20240501 19:57:43.145789 5295 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:43.145843 5829 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:43.150045 5694 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } }
I20240501 19:57:43.158135 5561 consensus_queue.cc:237] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } }
I20240501 19:57:43.158601 5429 consensus_queue.cc:237] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:43.160599 5962 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:43.160636 6095 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:43.160701 5829 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:43.160934 5694 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:43.161062 8829 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:43.161558 8343 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:43.161904 8343 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:43.162519 5295 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:43.162791 5561 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:43.163148 5829 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:43.163239 5695 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:43.163637 9084 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:43.163836 5960 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:43.164013 9084 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:43.164351 8573 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:43.164520 8829 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:43.164671 9083 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:43.165848 9083 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:43.167265 5295 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:43.168678 8969 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:43.169937 9083 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:43.170009 6096 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:43.170454 9083 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:43.171202 5962 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:43.172071 5296 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:43.172736 5695 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:43.173533 5138 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 0681575c2bb142e0915be3dc75858f87 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:43.174162 5694 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:43.174818 5147 catalog_manager.cc:5484] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:43.175884 5149 catalog_manager.cc:5484] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "9782508a47ba4403a39ca49fff7677fa" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 19:57:43.177079 5364 consensus_peers.cc:485] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0681575c2bb142e0915be3dc75858f87. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:43.177083 5827 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:43.179337 6095 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:43.179733 5561 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:43.180648 9098 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:43.182134 5962 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:43.182518 6095 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:43.185568 5429 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240501 19:57:43.186033 8970 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240501 19:57:43.187139 5561 consensus_queue.cc:237] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:43.187076 5829 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:43.190057 5429 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } }
I20240501 19:57:43.191102 5295 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
W20240501 19:57:43.191606 5497 consensus_peers.cc:485] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 98cbb51d815e4a82aa4e4f48bbeffd7b. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:43.192072 5429 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:43.192271 5294 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:43.192397 8970 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:43.192909 5694 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:43.193228 8966 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:43.193612 8966 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:43.194443 6097 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:43.197453 5829 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:43.197439 8966 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:43.197783 8970 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:43.199151 5962 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240501 19:57:43.199481 8970 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240501 19:57:43.199752 8966 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:43.201745 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:43.203605 5149 catalog_manager.cc:5484] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:43.203851 5695 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:43.204222 6095 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:43.204353 5429 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:43.203372 5296 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:43.207553 5962 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:43.212344 5829 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:43.277761 9134 ts_tablet_manager.cc:902] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:43.278182 9134 tablet_copy_client.cc:308] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:43.278628 5715 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet cb5540ce8d3343a68a8bf7d8fff00589 from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:34389)
I20240501 19:57:43.278726 5715 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet cb5540ce8d3343a68a8bf7d8fff00589 from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:34389: session id = 16737daa1d234f35826143a1aa29dc0b-cb5540ce8d3343a68a8bf7d8fff00589
I20240501 19:57:43.279384 5715 tablet_copy_source_session.cc:205] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:43.282466 9134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5540ce8d3343a68a8bf7d8fff00589. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:43.284642 9134 tablet_copy_client.cc:783] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:43.284818 9134 tablet_copy_client.cc:647] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:43.285980 9134 tablet_copy_client.cc:523] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:43.287179 9134 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:43.304080 9134 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:43.304344 9134 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:43.304520 9134 ts_tablet_manager.cc:1367] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.017s user 0.015s sys 0.000s
I20240501 19:57:43.304888 9134 raft_consensus.cc:348] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:43.305032 9134 raft_consensus.cc:729] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:43.305100 9134 consensus_queue.cc:260] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:43.305347 9134 ts_tablet_manager.cc:1398] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 19:57:43.305742 5715 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-cb5540ce8d3343a68a8bf7d8fff00589 received from {username='slave'} at 127.20.210.10:34389
I20240501 19:57:43.305820 5715 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-cb5540ce8d3343a68a8bf7d8fff00589 on tablet cb5540ce8d3343a68a8bf7d8fff00589 with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:43.574106 9134 ts_tablet_manager.cc:902] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
I20240501 19:57:43.574580 9134 tablet_copy_client.cc:308] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:35469
I20240501 19:57:43.578102 5450 tablet_copy_service.cc:140] P 9782508a47ba4403a39ca49fff7677fa: Received BeginTabletCopySession request for tablet 0681575c2bb142e0915be3dc75858f87 from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:38539)
I20240501 19:57:43.578214 5450 tablet_copy_service.cc:161] P 9782508a47ba4403a39ca49fff7677fa: Beginning new tablet copy session on tablet 0681575c2bb142e0915be3dc75858f87 from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:38539: session id = 16737daa1d234f35826143a1aa29dc0b-0681575c2bb142e0915be3dc75858f87
I20240501 19:57:43.579330 5450 tablet_copy_source_session.cc:205] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:43.580335 9134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0681575c2bb142e0915be3dc75858f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:43.582366 9134 tablet_copy_client.cc:783] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:43.582513 9134 tablet_copy_client.cc:647] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:43.583680 9134 tablet_copy_client.cc:523] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:43.584559 9134 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:43.599855 9134 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:43.600071 9134 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:43.600231 9134 ts_tablet_manager.cc:1367] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
I20240501 19:57:43.600564 9134 raft_consensus.cc:348] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:43.600670 9134 raft_consensus.cc:729] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:43.600775 9134 consensus_queue.cc:260] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:43.601047 9134 ts_tablet_manager.cc:1398] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:43.601529 5450 tablet_copy_service.cc:342] P 9782508a47ba4403a39ca49fff7677fa: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-0681575c2bb142e0915be3dc75858f87 received from {username='slave'} at 127.20.210.10:38539
I20240501 19:57:43.601622 5450 tablet_copy_service.cc:433] P 9782508a47ba4403a39ca49fff7677fa: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-0681575c2bb142e0915be3dc75858f87 on tablet 0681575c2bb142e0915be3dc75858f87 with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:43.644055 9134 ts_tablet_manager.cc:902] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:43.644474 9134 tablet_copy_client.cc:308] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:43.644860 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:45619)
I20240501 19:57:43.644968 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet 98cbb51d815e4a82aa4e4f48bbeffd7b from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:45619: session id = 16737daa1d234f35826143a1aa29dc0b-98cbb51d815e4a82aa4e4f48bbeffd7b
I20240501 19:57:43.645814 5583 tablet_copy_source_session.cc:205] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:43.646587 9134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:43.648150 9134 tablet_copy_client.cc:783] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:43.648294 9134 tablet_copy_client.cc:647] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:43.649259 9134 tablet_copy_client.cc:523] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:43.650075 9134 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:43.665194 9134 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:43.665398 9134 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:43.665537 9134 ts_tablet_manager.cc:1367] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20240501 19:57:43.665834 9134 raft_consensus.cc:348] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:43.665932 9134 raft_consensus.cc:729] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:43.666023 9134 consensus_queue.cc:260] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:43.666299 9134 ts_tablet_manager.cc:1398] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 19:57:43.666824 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-98cbb51d815e4a82aa4e4f48bbeffd7b received from {username='slave'} at 127.20.210.10:45619
I20240501 19:57:43.666891 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-98cbb51d815e4a82aa4e4f48bbeffd7b on tablet 98cbb51d815e4a82aa4e4f48bbeffd7b with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:43.698004 7684 raft_consensus.cc:1203] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:44.001374 7684 raft_consensus.cc:1203] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:44.134088 7684 raft_consensus.cc:1203] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240501 19:57:44.311367 8891 raft_consensus.cc:1050] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:44.311787 8891 consensus_queue.cc:237] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:44.313378 5962 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.313481 5429 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.313535 5296 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.313560 6097 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.313855 5829 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.313855 9223 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.313947 7684 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.314465 9219 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.314663 9221 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.314903 9217 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.315102 5561 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.315479 9222 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.315842 9222 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.316340 9219 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.318305 9222 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.318686 5962 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.319015 5561 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.321234 5829 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.321823 5145 catalog_manager.cc:5484] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.324120 5429 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.326092 5296 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.327709 6097 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.329316 7684 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.334789 5694 consensus_queue.cc:237] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:44.336503 5561 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.336513 6097 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.336849 5429 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.336853 9217 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.337040 7684 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.337129 9223 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.337296 9217 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.337422 9222 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.337709 5962 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.339417 9220 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.339917 9220 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.340147 5296 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.340341 7684 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.340970 9223 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.342393 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet cb5540ce8d3343a68a8bf7d8fff00589 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:44.342301 5296 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.342623 5429 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.342679 6097 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.343276 5145 catalog_manager.cc:5484] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b reported cstate change: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.345638 5809 tablet_service.cc:1514] Processing DeleteTablet for tablet cb5540ce8d3343a68a8bf7d8fff00589 with delete_type TABLET_DATA_TOMBSTONED (TS 1865147cd5a346ad8549f18fa8cc26c3 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:35230
I20240501 19:57:44.345973 9246 tablet_replica.cc:330] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: stopping tablet replica
I20240501 19:57:44.346129 9246 raft_consensus.cc:2229] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:44.346414 9246 raft_consensus.cc:2258] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:44.346344 5561 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.347157 9246 ts_tablet_manager.cc:1875] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:44.348995 5962 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.350093 9246 ts_tablet_manager.cc:1888] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:44.351640 9246 log.cc:1196] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/wals/cb5540ce8d3343a68a8bf7d8fff00589
I20240501 19:57:44.352170 5139 catalog_manager.cc:4830] TS 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499): tablet cb5540ce8d3343a68a8bf7d8fff00589 (table rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]) successfully deleted
I20240501 19:57:44.381891 9098 raft_consensus.cc:1050] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:44.382407 9098 consensus_queue.cc:237] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:44.384027 5962 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.384166 5561 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.384403 5296 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.384855 9254 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.385003 7684 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.385010 9253 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.385171 9097 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.385712 9097 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.386678 5694 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.386807 5829 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.387559 9253 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.387712 9097 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.387809 6097 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.388173 9082 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.388384 9097 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.389473 6097 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.389658 5829 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.391392 5962 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.391927 5296 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.392421 5694 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.393676 5145 catalog_manager.cc:5484] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9782508a47ba4403a39ca49fff7677fa" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.394554 5561 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.400023 7684 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.529771 9216 raft_consensus.cc:1050] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:44.530318 9216 consensus_queue.cc:237] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:44.532079 6097 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.532079 5296 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.532429 5962 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.532475 9215 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.532631 8970 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.533428 9265 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.533639 5426 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.534312 7684 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.534600 5829 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.534642 8970 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.535241 5694 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240501 19:57:44.535650 8970 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.535939 9216 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.536064 8970 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240501 19:57:44.536981 9216 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.537639 5426 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.539235 5296 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.539274 7684 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.540726 6097 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.541563 5153 catalog_manager.cc:5484] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.542222 5694 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.542543 5962 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.546108 5829 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.549816 5561 consensus_queue.cc:237] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:44.551450 5426 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.551479 7684 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.551726 6097 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.552150 9272 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.552201 5962 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.552451 9216 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.552613 9272 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.552886 9216 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.554869 9272 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.555326 5426 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.556795 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:44.557385 5145 catalog_manager.cc:5484] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.557852 5962 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.558774 5694 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.559270 9215 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.559478 5296 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.559818 5809 tablet_service.cc:1514] Processing DeleteTablet for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b with delete_type TABLET_DATA_TOMBSTONED (TS 1865147cd5a346ad8549f18fa8cc26c3 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:35230
I20240501 19:57:44.559979 9246 tablet_replica.cc:330] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3: stopping tablet replica
I20240501 19:57:44.560078 9246 raft_consensus.cc:2229] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:44.560220 9246 raft_consensus.cc:2258] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:44.560106 5694 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.560762 9215 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.562451 9246 ts_tablet_manager.cc:1875] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:44.562983 7683 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.564736 6097 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.566056 9246 ts_tablet_manager.cc:1888] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:44.566136 9246 log.cc:1196] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 1865147cd5a346ad8549f18fa8cc26c3: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/wals/98cbb51d815e4a82aa4e4f48bbeffd7b
I20240501 19:57:44.568593 5296 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.569183 5139 catalog_manager.cc:4830] TS 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499): tablet 98cbb51d815e4a82aa4e4f48bbeffd7b (table rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]) successfully deleted
I20240501 19:57:44.788724 5426 consensus_queue.cc:237] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:44.790270 6097 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.790310 5296 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.790642 5962 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.790719 9255 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.791023 5561 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.791366 5694 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.791803 9097 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.791944 9254 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.792268 7684 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240501 19:57:44.792295 9254 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.792408 9255 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.793273 9082 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240501 19:57:44.794273 9254 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.795840 7684 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.796164 5694 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.796336 5561 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.798134 5145 catalog_manager.cc:5484] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b reported cstate change: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New cstate: current_term: 1 leader_uuid: "9782508a47ba4403a39ca49fff7677fa" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.798354 5962 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.800117 5138 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0681575c2bb142e0915be3dc75858f87 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:44.800292 5295 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.800745 6097 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:44.803259 5809 tablet_service.cc:1514] Processing DeleteTablet for tablet 0681575c2bb142e0915be3dc75858f87 with delete_type TABLET_DATA_TOMBSTONED (TS 1865147cd5a346ad8549f18fa8cc26c3 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:35230
I20240501 19:57:44.803485 9246 tablet_replica.cc:330] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: stopping tablet replica
I20240501 19:57:44.803593 9246 raft_consensus.cc:2229] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:44.803853 9246 raft_consensus.cc:2258] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:44.804634 9246 ts_tablet_manager.cc:1875] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:44.806521 9246 ts_tablet_manager.cc:1888] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240501 19:57:44.806594 9246 log.cc:1196] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-4/wal/wals/0681575c2bb142e0915be3dc75858f87
I20240501 19:57:44.807407 5139 catalog_manager.cc:4830] TS 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499): tablet 0681575c2bb142e0915be3dc75858f87 (table rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]) successfully deleted
I20240501 19:57:45.254194 7530 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:45.254949 5674 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:45.258587 5408 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:45.263021 7664 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:45.268018 5809 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:45.271430 7398 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:45.288384 5942 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:45.291833 5543 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:45.294122 6076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:45.306404 5276 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:45.405658 5674 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:45.405731 7664 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:45.406031 7530 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:45.415135 5408 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:45.417589 5543 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240501 19:57:45.425258 5809 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:45.426229 5276 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:45.427548 5942 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:45.433581 6076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:45.441722 7398 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:45.509552 5694 consensus_queue.cc:237] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:45.511442 5295 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.511448 5426 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 19:57:45.511806 5629 consensus_peers.cc:485] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bd8a8b5e92c74598a5399fe0ca8c048d. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:45.511847 9393 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.511950 7684 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.512344 9393 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.512523 6097 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.512540 5563 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.512655 9392 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.512840 5962 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.512887 9393 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.513826 9393 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.514411 9392 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.515606 9393 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.516013 5426 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.516039 5295 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.517923 7684 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.518600 5152 catalog_manager.cc:5484] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:45.520141 6097 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.520242 5563 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.523175 5962 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.531570 5563 consensus_queue.cc:237] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:45.533221 6097 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.533221 5295 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.533231 7418 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.533541 9265 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.533867 5962 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.533869 5694 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 19:57:45.534256 5497 consensus_peers.cc:485] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c6ba23a7b4d24e0aa077e47798e890b8. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:45.534288 9412 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.534490 8970 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.534803 8970 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.535315 9216 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.535409 5426 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.535801 8970 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.536998 9216 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.537583 5962 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.538120 5426 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.538520 7418 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.539917 5153 catalog_manager.cc:5484] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.540717 6097 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.542920 5694 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.543079 5295 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.552920 5563 consensus_queue.cc:237] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:45.554886 5295 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.554947 7684 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.555297 5962 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.555400 6097 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 19:57:45.555546 5497 consensus_peers.cc:485] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): Couldn't send request to peer 363f4ac4964f41a6ba0f98d1f487db83. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 98cbb51d815e4a82aa4e4f48bbeffd7b. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:45.555737 9265 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.556318 5694 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.556308 9265 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.557374 8970 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.557659 8970 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.557704 5426 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.557833 9216 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.558101 9265 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.559764 8970 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.561901 7684 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.562125 5295 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.564518 5153 catalog_manager.cc:5484] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b reported cstate change: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.565021 5694 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.565033 5962 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.567798 6097 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.575258 5426 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.575942 5295 consensus_queue.cc:237] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:45.577586 5563 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.577699 6097 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.578320 5426 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.578481 5962 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.578573 5694 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.578645 9146 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.578935 9433 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.579008 7418 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 19:57:45.579212 5231 consensus_peers.cc:485] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85d29eb6552945a7879ee1a7e12919f0. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:45.579239 9433 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.579380 9384 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.579612 9436 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.579725 9384 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.581959 9434 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.582515 5694 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.582517 5426 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.582695 7418 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.585098 5149 catalog_manager.cc:5484] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.585366 6097 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.585834 5563 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.588138 5962 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.595624 5962 consensus_queue.cc:237] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:45.597060 5563 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.597393 9451 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.597572 7418 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.597707 5829 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.597744 6097 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.597941 9451 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.598076 9452 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.598327 5295 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.598327 5426 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.599102 8899 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.599515 9453 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.600823 9454 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.602465 9453 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.602771 5295 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
W20240501 19:57:45.603564 5896 consensus_peers.cc:485] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c59efbd84dd847188714271f4ba09ecb. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:45.604504 6097 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.605966 7418 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.607846 5150 catalog_manager.cc:5484] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.607990 5426 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.608991 5829 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.609846 5563 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.617285 5694 consensus_queue.cc:237] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:45.619029 6095 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.619485 7684 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.619539 5426 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.619779 5962 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 19:57:45.619864 5629 consensus_peers.cc:485] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae -> Peer 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): Couldn't send request to peer 363f4ac4964f41a6ba0f98d1f487db83. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cb5540ce8d3343a68a8bf7d8fff00589. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:45.619879 5563 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.620021 9392 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.620167 9391 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.620325 9398 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.620455 9223 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.620712 5295 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.621028 9219 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.621171 9392 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.623082 9219 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.624374 5962 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.625169 5563 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.625144 6095 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.627116 7684 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.628073 5429 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.629521 5145 catalog_manager.cc:5484] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.630055 5295 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.639504 5694 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:45.641229 5296 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.641229 5563 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.641731 9393 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.641904 7418 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.641996 5829 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.642201 6095 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.642640 9393 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 19:57:45.642812 5629 consensus_peers.cc:485] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f0df1e5e85294f60b8de22053dd577f7. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:45.643028 9223 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.643381 9223 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.644613 9398 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.645282 5429 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.645637 9223 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.646399 9393 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.647384 5296 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.648341 6095 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.649406 5829 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.650609 5150 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.652631 5429 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.652971 5563 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.656585 7418 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.663681 5829 consensus_queue.cc:237] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:45.665619 5296 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.665692 5429 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.666065 9500 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.666111 7550 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.666267 9499 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.666605 5694 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.666626 9499 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.666783 6095 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.666903 5563 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.667344 9499 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.667629 9499 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.667903 9499 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.669910 9498 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
W20240501 19:57:45.671332 5762 consensus_peers.cc:485] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a713bc0643d4bdf91f669dde94f63f8. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:45.671352 5563 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.672483 5154 catalog_manager.cc:5484] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:45.672793 7550 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.672830 5296 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.674278 5429 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.675544 5694 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.680312 6095 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.684267 5296 consensus_queue.cc:237] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:45.685773 5563 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.685860 5429 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.685914 7550 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.686041 5695 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.686125 6095 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.686311 9436 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 19:57:45.686491 5231 consensus_peers.cc:485] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 -> Peer 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): Couldn't send request to peer 363f4ac4964f41a6ba0f98d1f487db83. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 95e8db3b5d81468098c01c32c4d22e80. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:45.686540 9433 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.686692 5829 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.686844 9384 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.687052 9146 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.687202 9435 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.687407 9433 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.689788 9433 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.690284 6095 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.690608 5695 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.690771 5563 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.691900 7550 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.692449 5429 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.694711 5829 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:45.694898 5152 catalog_manager.cc:5484] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:45.704355 5695 consensus_queue.cc:237] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:45.705979 6095 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.706065 5296 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.706058 7418 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 19:57:45.706066 5629 consensus_peers.cc:485] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f27a79cf3a044383918f5d6f600a0fd9. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:45.706382 5429 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.706452 5829 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.706471 9223 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.706591 9391 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.706830 9392 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.707803 5563 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.707847 9223 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.708141 9393 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.708262 9392 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.709553 9391 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.709931 5296 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.710012 5829 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.712242 6095 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.712941 5154 catalog_manager.cc:5484] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.715466 7418 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.715438 5429 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.717387 5563 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.723958 5829 consensus_queue.cc:237] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:45.725534 5563 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.725816 5296 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.725991 9498 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.726223 9500 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.726307 6097 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 19:57:45.726405 5762 consensus_peers.cc:485] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e17a6ae467bf42abac714a94e1ec28e5. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:45.726584 5695 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.727080 9497 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.727452 9499 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.727799 5429 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.728215 9500 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.728266 7418 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.728597 9500 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.730129 9500 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.730636 5695 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.730773 5429 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.732964 5563 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.733316 5149 catalog_manager.cc:5484] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:45.734798 6097 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.735428 7418 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.736311 5296 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:45.744666 5829 consensus_queue.cc:237] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:45.746001 6095 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.746044 7418 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.746206 5563 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 19:57:45.746255 5762 consensus_peers.cc:485] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8a84349ba37496289351ac689f77eab. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:45.746520 9497 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.746836 9499 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.746937 5429 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.747074 5695 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.747176 5296 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:45.747278 9274 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.747529 9499 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.747674 9497 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.748523 9496 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:45.749725 9499 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.750146 5296 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.751109 6095 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.751415 5563 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.752308 5429 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.752424 7418 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.754135 5145 catalog_manager.cc:5484] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.758607 5695 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:45.906066 9618 ts_tablet_manager.cc:902] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:45.906641 9618 tablet_copy_client.cc:308] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:45.907167 5715 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet bd8a8b5e92c74598a5399fe0ca8c048d from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:49559)
I20240501 19:57:45.907263 5715 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet bd8a8b5e92c74598a5399fe0ca8c048d from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:49559: session id = 143c54da1456472dadbf1d5d65e0a6f6-bd8a8b5e92c74598a5399fe0ca8c048d
I20240501 19:57:45.908108 5715 tablet_copy_source_session.cc:205] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:45.909014 9618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd8a8b5e92c74598a5399fe0ca8c048d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:45.910919 9618 tablet_copy_client.cc:783] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:45.911078 9618 tablet_copy_client.cc:647] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:45.912187 9618 tablet_copy_client.cc:523] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:45.913076 9618 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:45.930968 9618 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:45.931211 9618 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:45.931360 9618 ts_tablet_manager.cc:1367] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.018s user 0.017s sys 0.001s
I20240501 19:57:45.931726 9618 raft_consensus.cc:348] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:45.931833 9618 raft_consensus.cc:729] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:45.931931 9618 consensus_queue.cc:260] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:45.932209 9618 ts_tablet_manager.cc:1398] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:45.932703 5715 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-bd8a8b5e92c74598a5399fe0ca8c048d received from {username='slave'} at 127.20.210.9:49559
I20240501 19:57:45.932782 5715 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-bd8a8b5e92c74598a5399fe0ca8c048d on tablet bd8a8b5e92c74598a5399fe0ca8c048d with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:46.024976 9624 ts_tablet_manager.cc:902] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:46.025457 9624 tablet_copy_client.cc:308] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:46.025919 5715 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet cb5540ce8d3343a68a8bf7d8fff00589 from peer 363f4ac4964f41a6ba0f98d1f487db83 ({username='slave'} at 127.20.210.8:53017)
I20240501 19:57:46.026034 5715 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet cb5540ce8d3343a68a8bf7d8fff00589 from peer 363f4ac4964f41a6ba0f98d1f487db83 at {username='slave'} at 127.20.210.8:53017: session id = 363f4ac4964f41a6ba0f98d1f487db83-cb5540ce8d3343a68a8bf7d8fff00589
I20240501 19:57:46.026966 5715 tablet_copy_source_session.cc:205] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:46.027885 9624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5540ce8d3343a68a8bf7d8fff00589. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:46.029882 9624 tablet_copy_client.cc:783] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:46.030031 9624 tablet_copy_client.cc:647] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:46.031131 9624 tablet_copy_client.cc:523] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:46.031996 9634 ts_tablet_manager.cc:902] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:46.032438 9624 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap starting.
I20240501 19:57:46.032912 9634 tablet_copy_client.cc:308] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:46.033424 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet c6ba23a7b4d24e0aa077e47798e890b8 from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:45619)
I20240501 19:57:46.033520 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet c6ba23a7b4d24e0aa077e47798e890b8 from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:45619: session id = 16737daa1d234f35826143a1aa29dc0b-c6ba23a7b4d24e0aa077e47798e890b8
I20240501 19:57:46.034432 5583 tablet_copy_source_session.cc:205] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:46.035410 9634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ba23a7b4d24e0aa077e47798e890b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:46.037453 9634 tablet_copy_client.cc:783] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:46.037654 9634 tablet_copy_client.cc:647] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:46.038769 9634 tablet_copy_client.cc:523] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:46.039647 9634 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:46.043972 9618 ts_tablet_manager.cc:902] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285)
I20240501 19:57:46.044420 9618 tablet_copy_client.cc:308] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:46285
I20240501 19:57:46.045264 5317 tablet_copy_service.cc:140] P ad52038f1d38489ea464a8fbc72daac0: Received BeginTabletCopySession request for tablet 85d29eb6552945a7879ee1a7e12919f0 from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:56127)
I20240501 19:57:46.045365 5317 tablet_copy_service.cc:161] P ad52038f1d38489ea464a8fbc72daac0: Beginning new tablet copy session on tablet 85d29eb6552945a7879ee1a7e12919f0 from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:56127: session id = 143c54da1456472dadbf1d5d65e0a6f6-85d29eb6552945a7879ee1a7e12919f0
I20240501 19:57:46.046164 5317 tablet_copy_source_session.cc:205] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:46.046996 9618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d29eb6552945a7879ee1a7e12919f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:46.049031 9618 tablet_copy_client.cc:783] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:46.049166 9618 tablet_copy_client.cc:647] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:46.051496 9618 tablet_copy_client.cc:523] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:46.052604 9618 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:46.057560 9624 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:46.057837 9624 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap complete.
I20240501 19:57:46.057991 9624 ts_tablet_manager.cc:1367] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent bootstrapping tablet: real 0.026s user 0.022s sys 0.000s
I20240501 19:57:46.058404 9624 raft_consensus.cc:348] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:46.058531 9624 raft_consensus.cc:729] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 363f4ac4964f41a6ba0f98d1f487db83, State: Initialized, Role: LEARNER
I20240501 19:57:46.058632 9624 consensus_queue.cc:260] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:46.059212 9624 ts_tablet_manager.cc:1398] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 19:57:46.059630 5715 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-cb5540ce8d3343a68a8bf7d8fff00589 received from {username='slave'} at 127.20.210.8:53017
I20240501 19:57:46.059700 5715 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-cb5540ce8d3343a68a8bf7d8fff00589 on tablet cb5540ce8d3343a68a8bf7d8fff00589 with peer 363f4ac4964f41a6ba0f98d1f487db83
I20240501 19:57:46.065496 9634 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:46.065781 9634 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:46.065944 9634 ts_tablet_manager.cc:1367] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.026s user 0.024s sys 0.000s
I20240501 19:57:46.066362 9634 raft_consensus.cc:348] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:46.066555 9634 raft_consensus.cc:729] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:46.066712 9634 consensus_queue.cc:260] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:46.067173 9634 ts_tablet_manager.cc:1398] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:46.067792 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-c6ba23a7b4d24e0aa077e47798e890b8 received from {username='slave'} at 127.20.210.10:45619
I20240501 19:57:46.067862 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-c6ba23a7b4d24e0aa077e47798e890b8 on tablet c6ba23a7b4d24e0aa077e47798e890b8 with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:46.076629 9618 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:46.076802 9618 tablet_bootstrap.cc:492] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:46.076946 9618 ts_tablet_manager.cc:1367] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.024s user 0.021s sys 0.000s
I20240501 19:57:46.077243 9618 raft_consensus.cc:348] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:46.077358 9618 raft_consensus.cc:729] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:46.077448 9618 consensus_queue.cc:260] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:46.077687 9618 ts_tablet_manager.cc:1398] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:46.078069 5317 tablet_copy_service.cc:342] P ad52038f1d38489ea464a8fbc72daac0: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-85d29eb6552945a7879ee1a7e12919f0 received from {username='slave'} at 127.20.210.9:56127
I20240501 19:57:46.078147 5317 tablet_copy_service.cc:433] P ad52038f1d38489ea464a8fbc72daac0: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-85d29eb6552945a7879ee1a7e12919f0 on tablet 85d29eb6552945a7879ee1a7e12919f0 with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:46.113083 9624 ts_tablet_manager.cc:902] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83: Initiating tablet copy from peer ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285)
I20240501 19:57:46.113452 9624 tablet_copy_client.cc:308] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:46285
I20240501 19:57:46.113847 5317 tablet_copy_service.cc:140] P ad52038f1d38489ea464a8fbc72daac0: Received BeginTabletCopySession request for tablet 95e8db3b5d81468098c01c32c4d22e80 from peer 363f4ac4964f41a6ba0f98d1f487db83 ({username='slave'} at 127.20.210.8:58425)
I20240501 19:57:46.113947 5317 tablet_copy_service.cc:161] P ad52038f1d38489ea464a8fbc72daac0: Beginning new tablet copy session on tablet 95e8db3b5d81468098c01c32c4d22e80 from peer 363f4ac4964f41a6ba0f98d1f487db83 at {username='slave'} at 127.20.210.8:58425: session id = 363f4ac4964f41a6ba0f98d1f487db83-95e8db3b5d81468098c01c32c4d22e80
I20240501 19:57:46.114941 5317 tablet_copy_source_session.cc:205] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:46.116735 9624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95e8db3b5d81468098c01c32c4d22e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:46.118741 9624 tablet_copy_client.cc:783] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:46.118883 9624 tablet_copy_client.cc:647] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:46.120438 9624 tablet_copy_client.cc:523] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:46.121528 9624 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap starting.
I20240501 19:57:46.144358 9624 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:46.144562 9624 tablet_bootstrap.cc:492] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap complete.
I20240501 19:57:46.144681 9624 ts_tablet_manager.cc:1367] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent bootstrapping tablet: real 0.023s user 0.016s sys 0.004s
I20240501 19:57:46.145009 9624 raft_consensus.cc:348] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:46.145105 9624 raft_consensus.cc:729] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 363f4ac4964f41a6ba0f98d1f487db83, State: Initialized, Role: LEARNER
I20240501 19:57:46.145190 9624 consensus_queue.cc:260] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:46.145452 9624 ts_tablet_manager.cc:1398] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:46.145987 5317 tablet_copy_service.cc:342] P ad52038f1d38489ea464a8fbc72daac0: Request end of tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-95e8db3b5d81468098c01c32c4d22e80 received from {username='slave'} at 127.20.210.8:58425
I20240501 19:57:46.146049 5317 tablet_copy_service.cc:433] P ad52038f1d38489ea464a8fbc72daac0: ending tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-95e8db3b5d81468098c01c32c4d22e80 on tablet 95e8db3b5d81468098c01c32c4d22e80 with peer 363f4ac4964f41a6ba0f98d1f487db83
I20240501 19:57:46.148547 9624 ts_tablet_manager.cc:902] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:46.148939 9624 tablet_copy_client.cc:308] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:46.149348 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b from peer 363f4ac4964f41a6ba0f98d1f487db83 ({username='slave'} at 127.20.210.8:48011)
I20240501 19:57:46.149430 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet 98cbb51d815e4a82aa4e4f48bbeffd7b from peer 363f4ac4964f41a6ba0f98d1f487db83 at {username='slave'} at 127.20.210.8:48011: session id = 363f4ac4964f41a6ba0f98d1f487db83-98cbb51d815e4a82aa4e4f48bbeffd7b
I20240501 19:57:46.150099 5583 tablet_copy_source_session.cc:205] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:46.150802 9624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:46.152478 9624 tablet_copy_client.cc:783] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:46.152617 9624 tablet_copy_client.cc:647] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:46.153579 9624 tablet_copy_client.cc:523] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:46.154491 9624 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap starting.
I20240501 19:57:46.169693 9634 ts_tablet_manager.cc:902] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085)
I20240501 19:57:46.170145 9634 tablet_copy_client.cc:308] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.6:34085
I20240501 19:57:46.173905 5983 tablet_copy_service.cc:140] P 9544b0e566c54a62a6135f3bff47c4f2: Received BeginTabletCopySession request for tablet c59efbd84dd847188714271f4ba09ecb from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:55191)
I20240501 19:57:46.174011 5983 tablet_copy_service.cc:161] P 9544b0e566c54a62a6135f3bff47c4f2: Beginning new tablet copy session on tablet c59efbd84dd847188714271f4ba09ecb from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:55191: session id = 16737daa1d234f35826143a1aa29dc0b-c59efbd84dd847188714271f4ba09ecb
I20240501 19:57:46.174883 5983 tablet_copy_source_session.cc:205] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:46.175719 9634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c59efbd84dd847188714271f4ba09ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:46.177704 9634 tablet_copy_client.cc:783] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:46.177856 9634 tablet_copy_client.cc:647] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:46.178957 9634 tablet_copy_client.cc:523] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:46.180066 9634 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:46.193940 9624 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:46.194159 9624 tablet_bootstrap.cc:492] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap complete.
I20240501 19:57:46.194326 9624 ts_tablet_manager.cc:1367] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83: Time spent bootstrapping tablet: real 0.040s user 0.022s sys 0.004s
I20240501 19:57:46.194710 9624 raft_consensus.cc:348] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:46.194841 9624 raft_consensus.cc:729] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 363f4ac4964f41a6ba0f98d1f487db83, State: Initialized, Role: LEARNER
I20240501 19:57:46.194927 9624 consensus_queue.cc:260] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:46.195528 9624 ts_tablet_manager.cc:1398] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:46.196697 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-98cbb51d815e4a82aa4e4f48bbeffd7b received from {username='slave'} at 127.20.210.8:48011
I20240501 19:57:46.196786 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-98cbb51d815e4a82aa4e4f48bbeffd7b on tablet 98cbb51d815e4a82aa4e4f48bbeffd7b with peer 363f4ac4964f41a6ba0f98d1f487db83
I20240501 19:57:46.204864 9634 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:46.217609 9618 ts_tablet_manager.cc:902] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:46.218032 9618 tablet_copy_client.cc:308] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:46.218432 5715 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet f0df1e5e85294f60b8de22053dd577f7 from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:49559)
I20240501 19:57:46.218521 5715 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet f0df1e5e85294f60b8de22053dd577f7 from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:49559: session id = 143c54da1456472dadbf1d5d65e0a6f6-f0df1e5e85294f60b8de22053dd577f7
I20240501 19:57:46.219234 5715 tablet_copy_source_session.cc:205] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:46.219920 9618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0df1e5e85294f60b8de22053dd577f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:46.221495 9618 tablet_copy_client.cc:783] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:46.221639 9618 tablet_copy_client.cc:647] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:46.222401 9634 tablet_bootstrap.cc:492] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:46.222573 9634 ts_tablet_manager.cc:1367] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.043s user 0.025s sys 0.000s
I20240501 19:57:46.222767 9618 tablet_copy_client.cc:523] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:46.223026 9634 raft_consensus.cc:348] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:46.223160 9634 raft_consensus.cc:729] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:46.223268 9634 consensus_queue.cc:260] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:46.223610 9634 ts_tablet_manager.cc:1398] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:46.223939 9618 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:46.224141 5983 tablet_copy_service.cc:342] P 9544b0e566c54a62a6135f3bff47c4f2: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-c59efbd84dd847188714271f4ba09ecb received from {username='slave'} at 127.20.210.10:55191
I20240501 19:57:46.224220 5983 tablet_copy_service.cc:433] P 9544b0e566c54a62a6135f3bff47c4f2: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-c59efbd84dd847188714271f4ba09ecb on tablet c59efbd84dd847188714271f4ba09ecb with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:46.229283 9723 ts_tablet_manager.cc:902] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:46.229702 9723 tablet_copy_client.cc:308] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:46.230087 5715 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet f27a79cf3a044383918f5d6f600a0fd9 from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:49559)
I20240501 19:57:46.230165 5715 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet f27a79cf3a044383918f5d6f600a0fd9 from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:49559: session id = 143c54da1456472dadbf1d5d65e0a6f6-f27a79cf3a044383918f5d6f600a0fd9
I20240501 19:57:46.230912 5715 tablet_copy_source_session.cc:205] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:46.231760 9723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f27a79cf3a044383918f5d6f600a0fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:46.233829 9723 tablet_copy_client.cc:783] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:46.233996 9723 tablet_copy_client.cc:647] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:46.236138 9723 tablet_copy_client.cc:523] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:46.236979 9723 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:46.243669 9634 ts_tablet_manager.cc:902] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499)
I20240501 19:57:46.244096 9634 tablet_copy_client.cc:308] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.5:39499
I20240501 19:57:46.245718 9618 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:46.245937 9618 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:46.246078 9618 ts_tablet_manager.cc:1367] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.022s user 0.019s sys 0.000s
I20240501 19:57:46.246479 9618 raft_consensus.cc:348] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:46.246599 9618 raft_consensus.cc:729] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:46.246724 9618 consensus_queue.cc:260] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:46.246992 9618 ts_tablet_manager.cc:1398] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 19:57:46.247721 5715 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-f0df1e5e85294f60b8de22053dd577f7 received from {username='slave'} at 127.20.210.9:49559
I20240501 19:57:46.247802 5715 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-f0df1e5e85294f60b8de22053dd577f7 on tablet f0df1e5e85294f60b8de22053dd577f7 with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:46.248126 5849 tablet_copy_service.cc:140] P 1865147cd5a346ad8549f18fa8cc26c3: Received BeginTabletCopySession request for tablet 4a713bc0643d4bdf91f669dde94f63f8 from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:51267)
I20240501 19:57:46.248207 5849 tablet_copy_service.cc:161] P 1865147cd5a346ad8549f18fa8cc26c3: Beginning new tablet copy session on tablet 4a713bc0643d4bdf91f669dde94f63f8 from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:51267: session id = 16737daa1d234f35826143a1aa29dc0b-4a713bc0643d4bdf91f669dde94f63f8
I20240501 19:57:46.248997 5849 tablet_copy_source_session.cc:205] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:46.249716 9634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a713bc0643d4bdf91f669dde94f63f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:46.251298 9634 tablet_copy_client.cc:783] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:46.251441 9634 tablet_copy_client.cc:647] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:46.253594 9634 tablet_copy_client.cc:523] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:46.254446 9634 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:46.257772 9723 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:46.257972 9723 tablet_bootstrap.cc:492] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:46.258122 9723 ts_tablet_manager.cc:1367] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.021s user 0.017s sys 0.000s
I20240501 19:57:46.258496 9723 raft_consensus.cc:348] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:46.258598 9723 raft_consensus.cc:729] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:46.258684 9723 consensus_queue.cc:260] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:46.259076 9723 ts_tablet_manager.cc:1398] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.001s user 0.005s sys 0.000s
I20240501 19:57:46.259544 5715 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-f27a79cf3a044383918f5d6f600a0fd9 received from {username='slave'} at 127.20.210.9:49559
I20240501 19:57:46.259600 5715 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-f27a79cf3a044383918f5d6f600a0fd9 on tablet f27a79cf3a044383918f5d6f600a0fd9 with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:46.272717 9634 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:46.272925 9634 tablet_bootstrap.cc:492] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:46.273074 9634 ts_tablet_manager.cc:1367] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.019s user 0.015s sys 0.000s
I20240501 19:57:46.273381 9634 raft_consensus.cc:348] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:46.273468 9634 raft_consensus.cc:729] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:46.273537 9634 consensus_queue.cc:260] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:46.273739 9634 ts_tablet_manager.cc:1398] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:46.274116 5849 tablet_copy_service.cc:342] P 1865147cd5a346ad8549f18fa8cc26c3: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-4a713bc0643d4bdf91f669dde94f63f8 received from {username='slave'} at 127.20.210.10:51267
I20240501 19:57:46.274189 5849 tablet_copy_service.cc:433] P 1865147cd5a346ad8549f18fa8cc26c3: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-4a713bc0643d4bdf91f669dde94f63f8 on tablet 4a713bc0643d4bdf91f669dde94f63f8 with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:46.324998 9723 ts_tablet_manager.cc:902] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499)
I20240501 19:57:46.325409 9723 tablet_copy_client.cc:308] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.5:39499
I20240501 19:57:46.325845 5849 tablet_copy_service.cc:140] P 1865147cd5a346ad8549f18fa8cc26c3: Received BeginTabletCopySession request for tablet d8a84349ba37496289351ac689f77eab from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:33281)
I20240501 19:57:46.325933 5849 tablet_copy_service.cc:161] P 1865147cd5a346ad8549f18fa8cc26c3: Beginning new tablet copy session on tablet d8a84349ba37496289351ac689f77eab from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:33281: session id = 143c54da1456472dadbf1d5d65e0a6f6-d8a84349ba37496289351ac689f77eab
I20240501 19:57:46.326673 5849 tablet_copy_source_session.cc:205] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:46.327399 9723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8a84349ba37496289351ac689f77eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:46.328899 9723 tablet_copy_client.cc:783] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:46.329026 9723 tablet_copy_client.cc:647] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:46.329981 9723 tablet_copy_client.cc:523] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:46.331073 9723 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:46.335044 9634 ts_tablet_manager.cc:902] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499)
I20240501 19:57:46.335515 9634 tablet_copy_client.cc:308] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.5:39499
I20240501 19:57:46.335919 5849 tablet_copy_service.cc:140] P 1865147cd5a346ad8549f18fa8cc26c3: Received BeginTabletCopySession request for tablet e17a6ae467bf42abac714a94e1ec28e5 from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:51267)
I20240501 19:57:46.336020 5849 tablet_copy_service.cc:161] P 1865147cd5a346ad8549f18fa8cc26c3: Beginning new tablet copy session on tablet e17a6ae467bf42abac714a94e1ec28e5 from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:51267: session id = 16737daa1d234f35826143a1aa29dc0b-e17a6ae467bf42abac714a94e1ec28e5
I20240501 19:57:46.336807 5849 tablet_copy_source_session.cc:205] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:46.337625 9634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e17a6ae467bf42abac714a94e1ec28e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:46.339257 9634 tablet_copy_client.cc:783] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:46.339385 9634 tablet_copy_client.cc:647] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:46.340318 9634 tablet_copy_client.cc:523] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:46.341209 9634 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:46.353969 9723 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:46.354223 9723 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:46.354406 9723 ts_tablet_manager.cc:1367] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.023s user 0.019s sys 0.004s
I20240501 19:57:46.354787 9723 raft_consensus.cc:348] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:46.354943 9723 raft_consensus.cc:729] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:46.355046 9723 consensus_queue.cc:260] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:46.355393 9723 ts_tablet_manager.cc:1398] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:46.355903 5849 tablet_copy_service.cc:342] P 1865147cd5a346ad8549f18fa8cc26c3: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-d8a84349ba37496289351ac689f77eab received from {username='slave'} at 127.20.210.9:33281
I20240501 19:57:46.355964 5849 tablet_copy_service.cc:433] P 1865147cd5a346ad8549f18fa8cc26c3: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-d8a84349ba37496289351ac689f77eab on tablet d8a84349ba37496289351ac689f77eab with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:46.365059 9634 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:46.365293 9634 tablet_bootstrap.cc:492] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:46.365442 9634 ts_tablet_manager.cc:1367] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.024s user 0.021s sys 0.000s
I20240501 19:57:46.365893 9634 raft_consensus.cc:348] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:46.366009 9634 raft_consensus.cc:729] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:46.366098 9634 consensus_queue.cc:260] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:46.366356 9634 ts_tablet_manager.cc:1398] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 19:57:46.366828 5849 tablet_copy_service.cc:342] P 1865147cd5a346ad8549f18fa8cc26c3: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-e17a6ae467bf42abac714a94e1ec28e5 received from {username='slave'} at 127.20.210.10:51267
I20240501 19:57:46.366904 5849 tablet_copy_service.cc:433] P 1865147cd5a346ad8549f18fa8cc26c3: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-e17a6ae467bf42abac714a94e1ec28e5 on tablet e17a6ae467bf42abac714a94e1ec28e5 with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:46.416108 7684 raft_consensus.cc:1203] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:46.467139 7418 raft_consensus.cc:1203] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:46.492018 7550 raft_consensus.cc:1203] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:46.588308 7418 raft_consensus.cc:1203] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:46.625211 7684 raft_consensus.cc:1203] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:46.641408 7684 raft_consensus.cc:1203] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:46.667553 7550 raft_consensus.cc:1203] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:46.669426 7550 raft_consensus.cc:1203] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:46.695742 7550 raft_consensus.cc:1203] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:46.731676 7418 raft_consensus.cc:1203] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:46.757659 7684 raft_consensus.cc:1203] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:46.892983 9483 raft_consensus.cc:1050] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:46.893596 9483 consensus_queue.cc:237] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:46.896014 5296 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.896224 5429 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.896708 9805 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.897089 9223 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.898097 7684 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.898329 7550 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.899170 6095 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.899351 5962 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.899821 5561 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.901116 9805 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.902060 9888 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.902555 9890 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.902732 9805 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.902859 9223 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.906497 9888 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.907375 7550 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.908553 5296 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.908792 6095 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.910053 5962 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.911821 5429 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.911854 5148 catalog_manager.cc:5484] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 reported cstate change: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.914609 7684 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.917212 5561 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.924644 5695 consensus_queue.cc:237] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:46.926427 5561 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:46.926429 7684 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:46.926753 5296 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:46.926739 9890 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:46.926944 7550 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:46.927179 9890 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:46.927423 9887 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:46.927536 6095 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:46.927577 9223 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:46.928335 5962 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:46.928673 9223 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:46.928962 9223 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:46.929813 9223 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.931154 6095 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.932040 5296 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.932332 7684 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.932938 5561 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.934480 5150 catalog_manager.cc:5484] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:46.934732 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bd8a8b5e92c74598a5399fe0ca8c048d with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:46.934942 5962 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.936371 7549 raft_consensus.cc:1203] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:46.937837 5408 tablet_service.cc:1514] Processing DeleteTablet for tablet bd8a8b5e92c74598a5399fe0ca8c048d with delete_type TABLET_DATA_TOMBSTONED (TS 9782508a47ba4403a39ca49fff7677fa not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:43934
I20240501 19:57:46.938331 9923 tablet_replica.cc:330] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa: stopping tablet replica
I20240501 19:57:46.938689 9923 raft_consensus.cc:2229] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:46.939007 9923 raft_consensus.cc:2258] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:46.940168 9923 ts_tablet_manager.cc:1875] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:46.941174 7550 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:46.942381 9923 ts_tablet_manager.cc:1888] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:46.942493 9923 log.cc:1196] T bd8a8b5e92c74598a5399fe0ca8c048d P 9782508a47ba4403a39ca49fff7677fa: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/bd8a8b5e92c74598a5399fe0ca8c048d
I20240501 19:57:46.943194 5138 catalog_manager.cc:4830] TS 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469): tablet bd8a8b5e92c74598a5399fe0ca8c048d (table rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]) successfully deleted
I20240501 19:57:46.979679 9890 raft_consensus.cc:1050] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 to VOTER
I20240501 19:57:46.980095 9890 consensus_queue.cc:237] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:46.981787 6095 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.981807 5296 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.981863 5962 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.982105 5429 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.982280 7418 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.982342 9483 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.982465 5561 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.982614 7684 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:46.982828 9483 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.982954 9887 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.983042 9805 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.983211 9888 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.983330 9890 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.983597 9889 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:46.986281 9889 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:46.986635 5296 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:46.987174 7418 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:46.988914 5429 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:46.989106 7684 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:46.989897 5145 catalog_manager.cc:5484] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:46.992422 5962 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:46.994024 5561 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:46.997246 6095 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:46.998708 9216 raft_consensus.cc:1050] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:46.999203 9216 consensus_queue.cc:237] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:46.999804 5695 consensus_queue.cc:237] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:47.001307 6095 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.001375 5962 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.001559 5296 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.001960 9889 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.002007 6097 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.002146 9804 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.002171 5426 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.002562 9804 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.002647 9890 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.002724 5962 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.002893 9890 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.003121 9890 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.004326 7683 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.004576 7417 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.004676 9889 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.004737 5695 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.005090 7418 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.005210 9412 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.005376 7684 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.005551 5429 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.005760 5296 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.005942 9804 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.006104 9216 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.006114 9887 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.006217 9412 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.007427 9887 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.007879 9412 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.009317 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet cb5540ce8d3343a68a8bf7d8fff00589 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:47.010417 5429 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.011013 7684 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.011636 5960 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.011747 6097 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.012312 5296 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.013561 9959 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.014708 5153 catalog_manager.cc:5484] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.014799 5296 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.016902 7418 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.018565 5543 tablet_service.cc:1514] Processing DeleteTablet for tablet cb5540ce8d3343a68a8bf7d8fff00589 with delete_type TABLET_DATA_TOMBSTONED (TS f5287d5ae4d84bd692b0f7985cd94171 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:43142
I20240501 19:57:47.019230 6097 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.019290 7684 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.019585 9970 tablet_replica.cc:330] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171: stopping tablet replica
I20240501 19:57:47.019518 5150 catalog_manager.cc:5484] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.019874 9970 raft_consensus.cc:2229] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:47.020161 9970 raft_consensus.cc:2258] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:47.022015 5960 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.022182 5429 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.024518 5695 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.027348 7417 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.027820 9970 ts_tablet_manager.cc:1875] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:47.028666 5561 consensus_queue.cc:237] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:47.030066 9970 ts_tablet_manager.cc:1888] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:47.030180 9970 log.cc:1196] T cb5540ce8d3343a68a8bf7d8fff00589 P f5287d5ae4d84bd692b0f7985cd94171: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/cb5540ce8d3343a68a8bf7d8fff00589
I20240501 19:57:47.030304 7684 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.030634 6097 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.030656 9958 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.030881 5141 catalog_manager.cc:4830] TS f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275): tablet cb5540ce8d3343a68a8bf7d8fff00589 (table rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]) successfully deleted
I20240501 19:57:47.030956 9216 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.032269 5695 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.032330 5429 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.032585 5960 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.032608 9962 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.032759 9804 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.033012 9804 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.033469 7417 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.033788 9216 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.034605 9804 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.034953 5695 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.035218 5429 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.035445 5962 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.036710 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c6ba23a7b4d24e0aa077e47798e890b8 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:47.037587 6097 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.037684 7684 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.037915 5154 catalog_manager.cc:5484] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.040133 7417 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.041502 5276 tablet_service.cc:1514] Processing DeleteTablet for tablet c6ba23a7b4d24e0aa077e47798e890b8 with delete_type TABLET_DATA_TOMBSTONED (TS ad52038f1d38489ea464a8fbc72daac0 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:57992
I20240501 19:57:47.041965 9971 tablet_replica.cc:330] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0: stopping tablet replica
I20240501 19:57:47.042351 9971 raft_consensus.cc:2229] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:47.042757 9971 raft_consensus.cc:2258] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:47.044215 9971 ts_tablet_manager.cc:1875] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:47.046010 9971 ts_tablet_manager.cc:1888] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:47.046073 9971 log.cc:1196] T c6ba23a7b4d24e0aa077e47798e890b8 P ad52038f1d38489ea464a8fbc72daac0: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/c6ba23a7b4d24e0aa077e47798e890b8
I20240501 19:57:47.046576 5138 catalog_manager.cc:4830] TS ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285): tablet c6ba23a7b4d24e0aa077e47798e890b8 (table rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]) successfully deleted
I20240501 19:57:47.085872 9962 raft_consensus.cc:1050] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 to VOTER
I20240501 19:57:47.086370 9962 consensus_queue.cc:237] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:47.087837 7417 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.087888 5296 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.088346 6097 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.088533 9412 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.088850 5960 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.089329 5429 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.089392 9216 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.089550 9412 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.089828 7684 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.089855 9216 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.090201 9216 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.090474 9216 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.091686 5695 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.092070 9961 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.095082 9958 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.098198 5145 catalog_manager.cc:5484] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:47.099043 6097 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.099262 7684 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.100350 5960 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.102157 5296 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.103390 7417 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.104655 5695 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.104683 5429 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.170738 9837 raft_consensus.cc:1050] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:47.171317 9837 consensus_queue.cc:237] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:47.173380 5561 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.173460 5296 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.173857 9983 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.174084 5829 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.174178 5429 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.174595 6097 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.174764 9837 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.174932 9985 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.175254 7417 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.175187 9984 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.175565 9837 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.175824 7684 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.176231 9837 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.176539 9837 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.178375 9984 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.178778 6097 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.180968 5561 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.181300 5429 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.181793 5296 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.181964 7417 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.183370 5149 catalog_manager.cc:5484] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.184744 7684 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.186995 9498 raft_consensus.cc:1050] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:47.187582 9498 consensus_queue.cc:237] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:47.189108 7417 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.189592 5429 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.189790 6097 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.189899 5563 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.189913 5695 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.190752 5296 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.191172 9497 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.191388 10005 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.191483 7683 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.191558 9870 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.191864 9497 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.191998 9870 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.192307 9870 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.193815 9870 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.196702 9497 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.196992 5695 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.198699 7417 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.199097 5429 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.199525 6097 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.200582 5153 catalog_manager.cc:5484] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:47.202956 5296 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.205469 5829 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.206678 10004 raft_consensus.cc:1050] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:47.207206 10004 consensus_queue.cc:237] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:47.207928 5563 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.208786 5960 consensus_queue.cc:237] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:47.210160 7550 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.210453 7684 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.210788 5429 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.210664 7683 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.211385 6097 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.211947 7684 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.211905 9982 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.212513 5429 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.212898 9497 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.213208 9982 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.213243 9497 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.213706 5829 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.214177 9982 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.214377 7417 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.215310 9982 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.215554 9980 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.216084 5561 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.216382 6097 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.216454 10003 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.216756 5695 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.216781 5296 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.217110 9497 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.217167 5563 raft_consensus.cc:1261] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9544b0e566c54a62a6135f3bff47c4f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.217413 9497 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.217747 9980 consensus_queue.cc:1035] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.217703 9983 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.218847 9497 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.219163 9497 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.220095 6096 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.220314 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c59efbd84dd847188714271f4ba09ecb with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:47.220690 5827 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.221190 5429 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.222975 5829 consensus_queue.cc:237] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:47.223114 5154 catalog_manager.cc:5484] T c59efbd84dd847188714271f4ba09ecb P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "9544b0e566c54a62a6135f3bff47c4f2" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:47.224922 7684 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.226452 7417 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.227481 7418 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.227818 7684 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.227849 9870 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.228075 9887 raft_consensus.cc:1050] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:47.228353 9870 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.228528 9887 consensus_queue.cc:237] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:47.228688 5562 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.228994 6096 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.229933 10003 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.230297 10003 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.230680 5276 tablet_service.cc:1514] Processing DeleteTablet for tablet c59efbd84dd847188714271f4ba09ecb with delete_type TABLET_DATA_TOMBSTONED (TS ad52038f1d38489ea464a8fbc72daac0 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:57992
I20240501 19:57:47.230908 9971 tablet_replica.cc:330] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0: stopping tablet replica
I20240501 19:57:47.230916 5430 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.230989 5295 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.231071 9971 raft_consensus.cc:2229] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:47.230950 5563 raft_consensus.cc:2932] T c59efbd84dd847188714271f4ba09ecb P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.231330 9971 raft_consensus.cc:2258] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:47.231410 6096 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.231453 9888 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.231817 9888 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.233098 5562 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.233989 9971 ts_tablet_manager.cc:1875] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:47.235426 5426 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.235716 9889 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.235805 9497 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.236039 9889 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.237525 9971 ts_tablet_manager.cc:1888] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:47.237593 9971 log.cc:1196] T c59efbd84dd847188714271f4ba09ecb P ad52038f1d38489ea464a8fbc72daac0: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/c59efbd84dd847188714271f4ba09ecb
I20240501 19:57:47.238102 5138 catalog_manager.cc:4830] TS ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285): tablet c59efbd84dd847188714271f4ba09ecb (table rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]) successfully deleted
I20240501 19:57:47.238621 9497 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.240607 7684 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.241034 10005 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.241065 7549 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.242466 5139 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e17a6ae467bf42abac714a94e1ec28e5 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:47.243017 7684 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.243026 6096 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.243531 6095 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.243620 5147 catalog_manager.cc:5484] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b reported cstate change: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.245740 5562 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.245738 5560 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.246829 7418 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.246888 5829 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.247325 5695 raft_consensus.cc:1261] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.247395 9869 raft_consensus.cc:1050] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:47.247871 9869 consensus_queue.cc:237] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:47.248931 10003 consensus_queue.cc:1035] T e17a6ae467bf42abac714a94e1ec28e5 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.249066 9888 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.249323 9805 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.249439 9888 raft_consensus.cc:1050] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:47.249912 9888 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:47.250109 5426 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.250294 7418 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.250473 10037 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.250412 5695 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.250769 7549 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.251129 5960 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.251996 6096 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.252135 10035 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.252288 5562 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.252485 10035 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.252787 10035 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.253089 10035 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.253371 7550 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.254547 7418 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.254910 5562 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.254966 5430 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.255282 10035 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.255263 9889 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.255503 5695 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.255570 9805 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.256032 5295 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.256511 9805 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.256733 9889 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.257097 10035 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.257153 5827 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.258116 7418 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.258510 6095 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.259140 5153 catalog_manager.cc:5484] T e17a6ae467bf42abac714a94e1ec28e5 P 363f4ac4964f41a6ba0f98d1f487db83 reported cstate change: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.259627 9805 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.259753 5695 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.260865 5429 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.261905 7549 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.262123 5276 tablet_service.cc:1514] Processing DeleteTablet for tablet e17a6ae467bf42abac714a94e1ec28e5 with delete_type TABLET_DATA_TOMBSTONED (TS ad52038f1d38489ea464a8fbc72daac0 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:57992
I20240501 19:57:47.266690 9887 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.268621 5429 raft_consensus.cc:2932] T e17a6ae467bf42abac714a94e1ec28e5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.268716 5561 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.269455 9971 tablet_replica.cc:330] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0: stopping tablet replica
I20240501 19:57:47.269630 9971 raft_consensus.cc:2229] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:47.270136 5430 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.270584 9971 raft_consensus.cc:2258] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:47.270577 6097 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.271417 9971 ts_tablet_manager.cc:1875] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:47.272869 5296 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.272923 5295 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.273084 5154 catalog_manager.cc:5484] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.275377 9483 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.275736 9483 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.276050 9223 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.277701 9971 ts_tablet_manager.cc:1888] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:47.277974 9971 log.cc:1196] T e17a6ae467bf42abac714a94e1ec28e5 P ad52038f1d38489ea464a8fbc72daac0: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/e17a6ae467bf42abac714a94e1ec28e5
I20240501 19:57:47.278563 5138 catalog_manager.cc:4830] TS ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285): tablet e17a6ae467bf42abac714a94e1ec28e5 (table rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]) successfully deleted
I20240501 19:57:47.278671 5829 consensus_queue.cc:237] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:47.279249 9223 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.279567 5426 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.280596 6097 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.281154 9870 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.281883 6095 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.282981 7684 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.283814 7549 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.283818 10035 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.284432 9870 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.284768 9870 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.285758 5561 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.285811 7549 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.286379 7418 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.286473 5426 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.288383 10004 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.288542 10002 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.288537 5145 catalog_manager.cc:5484] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6 reported cstate change: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.288865 5960 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.289108 6097 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.290333 5139 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4a713bc0643d4bdf91f669dde94f63f8 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:47.291420 7684 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.291635 5295 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.291893 7549 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.292951 5562 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.293229 5694 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.293547 5426 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.293859 10002 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.295082 5561 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.295235 5426 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.297431 5561 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.302510 6097 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.304795 5694 raft_consensus.cc:1261] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.305025 5829 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.305037 7550 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.305104 7418 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.306866 10002 consensus_queue.cc:1035] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.307523 7550 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.308283 5694 raft_consensus.cc:2932] T 4a713bc0643d4bdf91f669dde94f63f8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:47.310842 5147 catalog_manager.cc:5484] T 4a713bc0643d4bdf91f669dde94f63f8 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:47.312122 5153 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.313717 7417 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.316028 5827 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.317315 5276 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a713bc0643d4bdf91f669dde94f63f8 with delete_type TABLET_DATA_TOMBSTONED (TS ad52038f1d38489ea464a8fbc72daac0 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:57992
I20240501 19:57:47.317520 9971 tablet_replica.cc:330] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0: stopping tablet replica
I20240501 19:57:47.317656 9971 raft_consensus.cc:2229] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:47.317835 9971 raft_consensus.cc:2258] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:47.318974 9971 ts_tablet_manager.cc:1875] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:47.320686 5295 consensus_queue.cc:237] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:47.321008 9971 ts_tablet_manager.cc:1888] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:47.321055 9971 log.cc:1196] T 4a713bc0643d4bdf91f669dde94f63f8 P ad52038f1d38489ea464a8fbc72daac0: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/4a713bc0643d4bdf91f669dde94f63f8
I20240501 19:57:47.321516 5138 catalog_manager.cc:4830] TS ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285): tablet 4a713bc0643d4bdf91f669dde94f63f8 (table rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]) successfully deleted
I20240501 19:57:47.322635 7417 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.323143 6097 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.323323 10035 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.323486 5695 consensus_queue.cc:237] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:47.323961 5960 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.324155 10035 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.324378 10036 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.324777 5561 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.325897 5561 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.326198 10031 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.327436 9890 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.328646 5295 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.329252 7550 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.329509 9890 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.329628 10034 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.329510 10036 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.330034 6097 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.330936 5138 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 85d29eb6552945a7879ee1a7e12919f0 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:47.331416 7417 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.331696 7550 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.332206 6097 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.332314 5562 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.333598 5694 raft_consensus.cc:1261] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.333906 5960 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.334291 9888 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.334343 5145 catalog_manager.cc:5484] T 85d29eb6552945a7879ee1a7e12919f0 P 143c54da1456472dadbf1d5d65e0a6f6 reported cstate change: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.334822 10033 consensus_queue.cc:1035] T 85d29eb6552945a7879ee1a7e12919f0 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.335783 5827 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.336596 5695 raft_consensus.cc:2932] T 85d29eb6552945a7879ee1a7e12919f0 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.336978 7550 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.337543 10041 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.337867 10041 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.337986 7418 raft_consensus.cc:1261] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.338973 9888 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.340703 5408 tablet_service.cc:1514] Processing DeleteTablet for tablet 85d29eb6552945a7879ee1a7e12919f0 with delete_type TABLET_DATA_TOMBSTONED (TS 9782508a47ba4403a39ca49fff7677fa not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:43934
I20240501 19:57:47.340963 9923 tablet_replica.cc:330] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa: stopping tablet replica
I20240501 19:57:47.340799 5295 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.341125 9923 raft_consensus.cc:2229] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:47.341369 9923 raft_consensus.cc:2258] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:47.341454 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f27a79cf3a044383918f5d6f600a0fd9 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:47.341737 5562 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.342226 9923 ts_tablet_manager.cc:1875] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:47.344069 6097 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.344489 9923 ts_tablet_manager.cc:1888] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:47.344560 9923 log.cc:1196] T 85d29eb6552945a7879ee1a7e12919f0 P 9782508a47ba4403a39ca49fff7677fa: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/85d29eb6552945a7879ee1a7e12919f0
I20240501 19:57:47.344576 5152 catalog_manager.cc:5484] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:47.345031 9887 consensus_queue.cc:1035] T f27a79cf3a044383918f5d6f600a0fd9 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.345500 5138 catalog_manager.cc:4830] TS 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469): tablet 85d29eb6552945a7879ee1a7e12919f0 (table rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]) successfully deleted
I20240501 19:57:47.345938 5694 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:47.345860 5827 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.347514 5408 tablet_service.cc:1514] Processing DeleteTablet for tablet f27a79cf3a044383918f5d6f600a0fd9 with delete_type TABLET_DATA_TOMBSTONED (TS 9782508a47ba4403a39ca49fff7677fa not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:43934
I20240501 19:57:47.348704 9923 tablet_replica.cc:330] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa: stopping tablet replica
I20240501 19:57:47.348884 9923 raft_consensus.cc:2229] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:47.348942 5562 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.349233 9923 raft_consensus.cc:2258] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:47.349244 9887 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.350173 9923 ts_tablet_manager.cc:1875] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:47.350844 6097 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.351033 5295 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.351636 9483 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.351811 9890 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.352401 9923 ts_tablet_manager.cc:1888] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:47.352505 9923 log.cc:1196] T f27a79cf3a044383918f5d6f600a0fd9 P 9782508a47ba4403a39ca49fff7677fa: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/f27a79cf3a044383918f5d6f600a0fd9
I20240501 19:57:47.353071 5138 catalog_manager.cc:4830] TS 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469): tablet f27a79cf3a044383918f5d6f600a0fd9 (table rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]) successfully deleted
I20240501 19:57:47.353245 7417 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.354602 10034 raft_consensus.cc:1050] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0: attempting to promote NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 to VOTER
I20240501 19:57:47.354616 9483 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.354979 6097 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.355091 10034 consensus_queue.cc:237] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:47.356019 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f0df1e5e85294f60b8de22053dd577f7 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:47.356799 7418 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.357210 7549 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.357194 5295 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.357681 5426 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.358002 10037 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.358141 10035 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.358407 10037 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.359419 5562 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.359596 5561 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.359942 10037 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.360476 5827 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.360805 10037 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.360942 6097 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.361270 10037 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.361461 5694 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.362416 10037 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.362567 5145 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.364418 7418 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.364616 7549 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.365015 5829 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.365451 5408 tablet_service.cc:1514] Processing DeleteTablet for tablet f0df1e5e85294f60b8de22053dd577f7 with delete_type TABLET_DATA_TOMBSTONED (TS 9782508a47ba4403a39ca49fff7677fa not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:43934
I20240501 19:57:47.365404 10037 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.365677 9923 tablet_replica.cc:330] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: stopping tablet replica
I20240501 19:57:47.365819 9923 raft_consensus.cc:2229] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:47.365793 9890 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.366036 9923 raft_consensus.cc:2258] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:47.366653 9890 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.366874 9923 ts_tablet_manager.cc:1875] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:47.366988 5829 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.367035 5561 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.367242 9890 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.368330 7418 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.368732 9923 ts_tablet_manager.cc:1888] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:47.368806 9923 log.cc:1196] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/f0df1e5e85294f60b8de22053dd577f7
I20240501 19:57:47.369264 5138 catalog_manager.cc:4830] TS 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469): tablet f0df1e5e85294f60b8de22053dd577f7 (table rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]) successfully deleted
I20240501 19:57:47.369930 5148 catalog_manager.cc:5484] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.369976 7549 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.373548 7548 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.373597 5426 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.375545 7548 raft_consensus.cc:2932] T f27a79cf3a044383918f5d6f600a0fd9 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.373276 7417 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.377626 6097 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.382499 5694 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.382958 5827 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.386555 5295 consensus_queue.cc:237] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:47.388213 5426 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.388228 5827 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.388422 7549 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.388567 10034 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.388741 10033 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.389004 6097 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.388989 10035 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.389498 10035 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.391351 5694 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.391672 10037 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.392005 10034 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.392640 5694 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.393540 5138 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 95e8db3b5d81468098c01c32c4d22e80 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:47.393596 5827 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.393973 7549 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.395584 5152 catalog_manager.cc:5484] T 95e8db3b5d81468098c01c32c4d22e80 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.395599 6097 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.397260 5426 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.397553 7417 raft_consensus.cc:1261] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.398532 10031 consensus_queue.cc:1035] T 95e8db3b5d81468098c01c32c4d22e80 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.400136 5543 tablet_service.cc:1514] Processing DeleteTablet for tablet 95e8db3b5d81468098c01c32c4d22e80 with delete_type TABLET_DATA_TOMBSTONED (TS f5287d5ae4d84bd692b0f7985cd94171 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:43142
I20240501 19:57:47.400331 9970 tablet_replica.cc:330] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171: stopping tablet replica
I20240501 19:57:47.400454 9970 raft_consensus.cc:2229] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:47.400665 9970 raft_consensus.cc:2258] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:47.402611 9970 ts_tablet_manager.cc:1875] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:47.404242 9970 ts_tablet_manager.cc:1888] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:47.404290 9970 log.cc:1196] T 95e8db3b5d81468098c01c32c4d22e80 P f5287d5ae4d84bd692b0f7985cd94171: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/95e8db3b5d81468098c01c32c4d22e80
I20240501 19:57:47.404664 5141 catalog_manager.cc:4830] TS f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275): tablet 95e8db3b5d81468098c01c32c4d22e80 (table rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]) successfully deleted
I20240501 19:57:47.405927 7417 raft_consensus.cc:2932] T 95e8db3b5d81468098c01c32c4d22e80 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:47.411454 5561 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b"
dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:52754
I20240501 19:57:47.411579 5561 raft_consensus.cc:595] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Received request to transfer leadership
I20240501 19:57:47.442505 5561 consensus_queue.cc:237] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:47.444334 5295 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:47.444334 5426 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:47.444563 5827 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:47.444830 9958 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:47.444903 7684 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:47.445439 9958 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:47.445582 5960 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:47.445582 6097 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:47.446041 9804 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 19:57:47.446166 5497 consensus_peers.cc:485] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6b6181bae544db0b6bbdc6bcd01a40d. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:47.446388 9804 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:47.446511 9962 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:47.446637 9961 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:47.448973 9962 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.449498 6097 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.450012 5426 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.451980 5295 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.452055 5827 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.452335 7684 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.452667 5149 catalog_manager.cc:5484] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:47.462805 5962 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.473163 10002 raft_consensus.cc:1050] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:47.473719 10002 consensus_queue.cc:237] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:47.475956 7549 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.475953 5561 consensus_queue.cc:237] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:47.476359 10004 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.476550 5426 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.476881 5295 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.477085 7417 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.477193 6097 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.477536 5694 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.477684 5295 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:47.477836 7684 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:47.478224 9960 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:47.478421 5562 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:47.478621 9804 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:47.479189 9498 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20240501 19:57:47.479454 5497 consensus_peers.cc:485] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3937a2d841440d793ba2637f1af060f. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:47.479502 9498 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.479624 10001 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.479835 9498 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.480340 9497 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.480732 5962 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:47.480736 6096 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:47.481185 9960 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:47.481500 9960 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:47.482937 5426 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:47.483228 5694 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:47.483294 9804 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:47.483399 10004 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:47.484902 9960 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:47.486749 10004 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.487164 5430 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.487602 9804 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.487525 7549 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.489842 6096 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.490057 5296 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.491365 5151 catalog_manager.cc:5484] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.491853 7684 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.492626 5145 catalog_manager.cc:5484] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:47.493474 5430 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.494197 6096 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.496232 5694 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.499840 5960 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.500502 7417 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.502632 5295 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:47.507261 5695 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.507498 5561 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.512207 5827 consensus_queue.cc:237] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:47.514374 5430 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.514372 5295 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.514587 6096 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.514681 7417 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.515007 9870 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.515110 10002 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.515269 10003 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.515385 10002 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.517110 5695 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.517490 10003 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.517943 7549 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:47.517498 9498 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.518110 5430 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.518515 10002 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.519721 5139 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d8a84349ba37496289351ac689f77eab with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:47.519887 5695 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.520200 5295 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.521226 5152 catalog_manager.cc:5484] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:47.521597 6096 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.524314 9216 raft_consensus.cc:979] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: : Instructing follower 16737daa1d234f35826143a1aa29dc0b to start an election
I20240501 19:57:47.524418 9216 raft_consensus.cc:1067] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Signalling peer 16737daa1d234f35826143a1aa29dc0b to start an election
I20240501 19:57:47.525039 7684 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b"
dest_uuid: "16737daa1d234f35826143a1aa29dc0b"
from {username='slave'} at 127.20.210.3:56133
I20240501 19:57:47.525220 7684 raft_consensus.cc:482] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240501 19:57:47.525346 7684 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:47.526404 7684 raft_consensus.cc:504] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:47.526620 5543 tablet_service.cc:1514] Processing DeleteTablet for tablet d8a84349ba37496289351ac689f77eab with delete_type TABLET_DATA_TOMBSTONED (TS f5287d5ae4d84bd692b0f7985cd94171 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:43142
I20240501 19:57:47.526861 9970 tablet_replica.cc:330] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171: stopping tablet replica
I20240501 19:57:47.526997 9970 raft_consensus.cc:2229] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:47.527202 9970 raft_consensus.cc:2258] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:47.527959 9970 ts_tablet_manager.cc:1875] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:47.527961 5694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "16737daa1d234f35826143a1aa29dc0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "4d4855c096ac4e1e936abf15834332ae"
I20240501 19:57:47.528056 5694 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:47.528465 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "16737daa1d234f35826143a1aa29dc0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "9544b0e566c54a62a6135f3bff47c4f2"
I20240501 19:57:47.528589 5962 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:47.528920 5694 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16737daa1d234f35826143a1aa29dc0b in term 2.
I20240501 19:57:47.529336 5295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "16737daa1d234f35826143a1aa29dc0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "ad52038f1d38489ea464a8fbc72daac0"
I20240501 19:57:47.529428 5295 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:47.529589 5962 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16737daa1d234f35826143a1aa29dc0b in term 2.
I20240501 19:57:47.529788 5430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "16737daa1d234f35826143a1aa29dc0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "9782508a47ba4403a39ca49fff7677fa"
I20240501 19:57:47.529892 5430 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:47.529970 9970 ts_tablet_manager.cc:1888] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:47.530040 9970 log.cc:1196] T d8a84349ba37496289351ac689f77eab P f5287d5ae4d84bd692b0f7985cd94171: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/d8a84349ba37496289351ac689f77eab
I20240501 19:57:47.530305 5295 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16737daa1d234f35826143a1aa29dc0b in term 2.
I20240501 19:57:47.530730 5141 catalog_manager.cc:4830] TS f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275): tablet d8a84349ba37496289351ac689f77eab (table rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]) successfully deleted
I20240501 19:57:47.530839 5430 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16737daa1d234f35826143a1aa29dc0b in term 2.
I20240501 19:57:47.530689 7417 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.531169 7616 leader_election.cc:302] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 8 voters: 5 yes votes; 0 no votes. yes voters: 16737daa1d234f35826143a1aa29dc0b, 4d4855c096ac4e1e936abf15834332ae, 9544b0e566c54a62a6135f3bff47c4f2, 9782508a47ba4403a39ca49fff7677fa, ad52038f1d38489ea464a8fbc72daac0; no voters:
I20240501 19:57:47.531416 9401 raft_consensus.cc:2781] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 2 FOLLOWER]: Leader election won for term 2
I20240501 19:57:47.531606 9401 raft_consensus.cc:686] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 2 LEADER]: Becoming Leader. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Running, Role: LEADER
I20240501 19:57:47.531934 9401 consensus_queue.cc:237] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:47.525486 7549 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:47.535326 5148 catalog_manager.cc:5484] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b reported cstate change: term changed from 1 to 2, leader changed from f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) to 16737daa1d234f35826143a1aa29dc0b (127.20.210.10). New cstate: current_term: 2 leader_uuid: "16737daa1d234f35826143a1aa29dc0b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:47.535742 7684 leader_election.cc:288] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [CANDIDATE]: Term 2 election: Requested vote from peers 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519), 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085), a881063bd86945a8871c7c102d8235c8 (127.20.210.7:39815), ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285), 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469), f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275), 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983)
I20240501 19:57:47.535765 5561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "16737daa1d234f35826143a1aa29dc0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "f5287d5ae4d84bd692b0f7985cd94171"
I20240501 19:57:47.535871 5561 raft_consensus.cc:3032] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Stepping down as leader of term 1
I20240501 19:57:47.535930 5561 raft_consensus.cc:729] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5287d5ae4d84bd692b0f7985cd94171, State: Running, Role: LEADER
I20240501 19:57:47.536077 5561 consensus_queue.cc:260] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:47.536306 5561 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:47.536947 6096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "16737daa1d234f35826143a1aa29dc0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "a881063bd86945a8871c7c102d8235c8"
I20240501 19:57:47.537062 6096 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:47.537123 5561 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16737daa1d234f35826143a1aa29dc0b in term 2.
I20240501 19:57:47.537959 6096 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16737daa1d234f35826143a1aa29dc0b in term 2.
I20240501 19:57:47.542588 7417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98cbb51d815e4a82aa4e4f48bbeffd7b" candidate_uuid: "16737daa1d234f35826143a1aa29dc0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "363f4ac4964f41a6ba0f98d1f487db83"
I20240501 19:57:47.542744 7417 raft_consensus.cc:3037] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Advancing to term 2
I20240501 19:57:47.543720 7417 raft_consensus.cc:2454] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16737daa1d234f35826143a1aa29dc0b in term 2.
I20240501 19:57:47.914848 10226 ts_tablet_manager.cc:902] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:47.915424 10226 tablet_copy_client.cc:308] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:47.915802 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet d6b6181bae544db0b6bbdc6bcd01a40d from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:42563)
I20240501 19:57:47.915887 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet d6b6181bae544db0b6bbdc6bcd01a40d from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:42563: session id = 143c54da1456472dadbf1d5d65e0a6f6-d6b6181bae544db0b6bbdc6bcd01a40d
I20240501 19:57:47.916518 5583 tablet_copy_source_session.cc:205] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:47.917277 10226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6b6181bae544db0b6bbdc6bcd01a40d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:47.918999 10226 tablet_copy_client.cc:783] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:47.919139 10226 tablet_copy_client.cc:647] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:47.920217 10226 tablet_copy_client.cc:523] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:47.921171 10226 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:47.924190 10232 ts_tablet_manager.cc:902] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:47.924680 10232 tablet_copy_client.cc:308] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:47.925072 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet d3937a2d841440d793ba2637f1af060f from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:42563)
I20240501 19:57:47.925163 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet d3937a2d841440d793ba2637f1af060f from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:42563: session id = 143c54da1456472dadbf1d5d65e0a6f6-d3937a2d841440d793ba2637f1af060f
I20240501 19:57:47.925853 5583 tablet_copy_source_session.cc:205] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:47.926549 10232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3937a2d841440d793ba2637f1af060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:47.943962 10232 tablet_copy_client.cc:783] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:47.944139 10232 tablet_copy_client.cc:647] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:47.945190 10232 tablet_copy_client.cc:523] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:47.946062 10232 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:47.961930 5561 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 2 FOLLOWER]: Refusing update from remote peer 16737daa1d234f35826143a1aa29dc0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:47.962512 10166 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:47.965888 10226 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:47.966179 10226 tablet_bootstrap.cc:492] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:47.966351 10226 ts_tablet_manager.cc:1367] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.045s user 0.026s sys 0.000s
I20240501 19:57:47.966725 10226 raft_consensus.cc:348] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:47.966849 10226 raft_consensus.cc:729] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:47.966980 10226 consensus_queue.cc:260] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:47.967885 10226 ts_tablet_manager.cc:1398] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:47.967950 10232 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:47.968329 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-d6b6181bae544db0b6bbdc6bcd01a40d received from {username='slave'} at 127.20.210.9:42563
I20240501 19:57:47.968426 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-d6b6181bae544db0b6bbdc6bcd01a40d on tablet d6b6181bae544db0b6bbdc6bcd01a40d with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:47.968266 10232 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:47.968860 10232 ts_tablet_manager.cc:1367] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.023s user 0.021s sys 0.000s
I20240501 19:57:47.969148 10232 raft_consensus.cc:348] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:47.969254 10232 raft_consensus.cc:729] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:47.969404 10232 consensus_queue.cc:260] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:47.969698 10232 ts_tablet_manager.cc:1398] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:47.970350 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-d3937a2d841440d793ba2637f1af060f received from {username='slave'} at 127.20.210.9:42563
I20240501 19:57:47.970445 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-d3937a2d841440d793ba2637f1af060f on tablet d3937a2d841440d793ba2637f1af060f with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:48.013394 7417 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83 [term 2 FOLLOWER]: Refusing update from remote peer 16737daa1d234f35826143a1aa29dc0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:48.013789 10166 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:48.028535 5426 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 2 FOLLOWER]: Refusing update from remote peer 16737daa1d234f35826143a1aa29dc0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:48.028857 10166 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:48.056046 5695 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 2 FOLLOWER]: Refusing update from remote peer 16737daa1d234f35826143a1aa29dc0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:48.056397 10166 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:48.059396 5295 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 2 FOLLOWER]: Refusing update from remote peer 16737daa1d234f35826143a1aa29dc0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:48.060040 10256 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:48.060487 6096 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 2 FOLLOWER]: Refusing update from remote peer 16737daa1d234f35826143a1aa29dc0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:48.062254 10256 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:48.062515 5962 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 2 FOLLOWER]: Refusing update from remote peer 16737daa1d234f35826143a1aa29dc0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240501 19:57:48.063045 10256 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:48.353343 7549 raft_consensus.cc:1203] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:48.512370 7549 raft_consensus.cc:1203] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:48.618829 7683 consensus_queue.cc:237] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 2.10, Last appended by leader: 9, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:48.620434 5296 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 2 FOLLOWER]: Refusing update from remote peer 16737daa1d234f35826143a1aa29dc0b: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 19:57:48.620462 7417 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83 [term 2 FOLLOWER]: Refusing update from remote peer 16737daa1d234f35826143a1aa29dc0b: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 19:57:48.620451 5430 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 2 FOLLOWER]: Refusing update from remote peer 16737daa1d234f35826143a1aa29dc0b: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 19:57:48.620656 5695 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 2 FOLLOWER]: Refusing update from remote peer 16737daa1d234f35826143a1aa29dc0b: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 19:57:48.621022 6096 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 2 FOLLOWER]: Refusing update from remote peer 16737daa1d234f35826143a1aa29dc0b: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 19:57:48.621047 10260 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:48.621101 5962 raft_consensus.cc:1261] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 2 FOLLOWER]: Refusing update from remote peer 16737daa1d234f35826143a1aa29dc0b: Log matching property violated. Preceding OpId in replica: term: 2 index: 10. Preceding OpId from leader: term: 2 index: 11. (index mismatch)
I20240501 19:57:48.621295 10303 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:48.621461 10255 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:48.621645 10256 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:48.621766 10304 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:48.621968 10302 consensus_queue.cc:1035] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:48.624156 10256 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b [term 2 LEADER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:48.625519 5962 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9544b0e566c54a62a6135f3bff47c4f2 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:48.625911 7417 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 363f4ac4964f41a6ba0f98d1f487db83 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:48.627396 5139 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:48.627374 5430 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 9782508a47ba4403a39ca49fff7677fa [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:48.627735 5154 catalog_manager.cc:5484] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 16737daa1d234f35826143a1aa29dc0b reported cstate change: config changed from index 9 to 11, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New cstate: current_term: 2 leader_uuid: "16737daa1d234f35826143a1aa29dc0b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:48.627909 5695 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P 4d4855c096ac4e1e936abf15834332ae [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:48.627909 5296 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P ad52038f1d38489ea464a8fbc72daac0 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:48.628952 6096 raft_consensus.cc:2932] T 98cbb51d815e4a82aa4e4f48bbeffd7b P a881063bd86945a8871c7c102d8235c8 [term 2 FOLLOWER]: Committing config change with OpId 2.11: config changed from index 9 to 11, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:48.634073 5543 tablet_service.cc:1514] Processing DeleteTablet for tablet 98cbb51d815e4a82aa4e4f48bbeffd7b with delete_type TABLET_DATA_TOMBSTONED (TS f5287d5ae4d84bd692b0f7985cd94171 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:43142
I20240501 19:57:48.634620 10311 tablet_replica.cc:330] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: stopping tablet replica
I20240501 19:57:48.634771 10311 raft_consensus.cc:2229] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:48.634939 10311 raft_consensus.cc:2258] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:48.635609 10311 ts_tablet_manager.cc:1875] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:48.637444 10311 ts_tablet_manager.cc:1888] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.10
I20240501 19:57:48.637506 10311 log.cc:1196] T 98cbb51d815e4a82aa4e4f48bbeffd7b P f5287d5ae4d84bd692b0f7985cd94171: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/98cbb51d815e4a82aa4e4f48bbeffd7b
I20240501 19:57:48.637941 5141 catalog_manager.cc:4830] TS f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275): tablet 98cbb51d815e4a82aa4e4f48bbeffd7b (table rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]) successfully deleted
I20240501 19:57:48.944706 9958 raft_consensus.cc:1050] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:48.945199 9958 consensus_queue.cc:237] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:48.947093 6096 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:48.947225 5827 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:48.947248 7683 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:48.947479 10352 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:48.947723 10350 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:48.947984 5296 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:48.948293 10352 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:48.948370 7549 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:48.948719 9961 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:48.948863 5430 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:48.950449 5960 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:48.950847 10352 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:48.951095 9961 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:48.951277 10285 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:48.951995 10352 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.953387 6096 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.954602 7549 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.954643 7683 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.956095 5827 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.957110 5149 catalog_manager.cc:5484] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:48.958912 5296 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.960966 5430 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.960966 5960 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.965448 5561 consensus_queue.cc:237] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:48.966817 7549 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:48.966948 5827 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:48.967077 7683 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:48.967283 6096 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:48.967464 5960 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:48.967438 10350 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:48.967622 10351 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:48.967731 10352 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:48.967954 10285 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:48.968068 9958 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:48.970078 5296 raft_consensus.cc:1261] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:48.970402 9958 consensus_queue.cc:1035] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:48.970124 10285 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.970710 6096 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.972421 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d6b6181bae544db0b6bbdc6bcd01a40d with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:48.972534 7683 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.972887 5827 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.973403 5148 catalog_manager.cc:5484] T d6b6181bae544db0b6bbdc6bcd01a40d P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.974119 7549 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.975502 5296 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.977082 5960 raft_consensus.cc:2932] T d6b6181bae544db0b6bbdc6bcd01a40d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:48.977880 5408 tablet_service.cc:1514] Processing DeleteTablet for tablet d6b6181bae544db0b6bbdc6bcd01a40d with delete_type TABLET_DATA_TOMBSTONED (TS 9782508a47ba4403a39ca49fff7677fa not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:43934
I20240501 19:57:48.978183 10362 tablet_replica.cc:330] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa: stopping tablet replica
I20240501 19:57:48.978358 10362 raft_consensus.cc:2229] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:48.978768 10362 raft_consensus.cc:2258] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:48.979365 10362 ts_tablet_manager.cc:1875] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:48.982007 10362 ts_tablet_manager.cc:1888] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:48.982138 10362 log.cc:1196] T d6b6181bae544db0b6bbdc6bcd01a40d P 9782508a47ba4403a39ca49fff7677fa: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/d6b6181bae544db0b6bbdc6bcd01a40d
I20240501 19:57:48.982677 5138 catalog_manager.cc:4830] TS 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469): tablet d6b6181bae544db0b6bbdc6bcd01a40d (table rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]) successfully deleted
I20240501 19:57:49.101768 10353 raft_consensus.cc:1050] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:49.102387 10353 consensus_queue.cc:237] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:49.104691 5960 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:49.104789 5695 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:49.104812 6096 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:49.105034 5296 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:49.105062 7549 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:49.105317 10374 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:49.105368 5430 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:49.105465 10352 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:49.105832 10352 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:49.105923 7683 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:49.106082 9958 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:49.106400 10351 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:49.106554 9958 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:49.108011 10285 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:49.110191 10352 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.111346 5960 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.111546 6096 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.112035 7549 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.113981 5430 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.114388 5695 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.114879 5145 catalog_manager.cc:5484] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.115638 5296 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.117380 7683 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.123901 5561 consensus_queue.cc:237] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:49.125373 7683 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:49.125553 5430 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:49.125952 10374 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:49.126051 6096 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:49.126084 7549 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:49.126498 5695 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:49.126498 10352 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:49.126812 10352 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:49.126974 10350 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:49.127115 9958 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:49.127386 5960 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:49.127863 10350 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:49.129294 10351 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.130021 5695 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.131489 7683 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.131676 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d3937a2d841440d793ba2637f1af060f with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:49.131858 7549 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.132501 5960 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.132601 5152 catalog_manager.cc:5484] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:49.136683 5276 tablet_service.cc:1514] Processing DeleteTablet for tablet d3937a2d841440d793ba2637f1af060f with delete_type TABLET_DATA_TOMBSTONED (TS ad52038f1d38489ea464a8fbc72daac0 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:57992
I20240501 19:57:49.136664 5430 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.137699 10387 tablet_replica.cc:330] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: stopping tablet replica
I20240501 19:57:49.137915 10387 raft_consensus.cc:2229] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:49.138360 6096 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:49.138221 10387 raft_consensus.cc:2258] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:49.139600 10387 ts_tablet_manager.cc:1875] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:49.141872 10387 ts_tablet_manager.cc:1888] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:49.141937 10387 log.cc:1196] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-0/wal/wals/d3937a2d841440d793ba2637f1af060f
I20240501 19:57:49.142446 5138 catalog_manager.cc:4830] TS ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285): tablet d3937a2d841440d793ba2637f1af060f (table rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]) successfully deleted
I20240501 19:57:49.389585 5942 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:49.392411 7530 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:49.399750 7664 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:49.401487 5543 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:49.401504 5809 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:49.402462 5674 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:49.406930 5408 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:49.407470 5276 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:49.418042 7398 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:49.437798 6076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:49.544937 5942 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:49.546939 7664 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:49.547448 7530 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:49.554668 5276 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:49.556185 5809 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:49.565735 5674 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:49.571270 7398 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:49.582816 6076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:49.583484 5543 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:49.596284 5408 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:49.641732 5695 consensus_queue.cc:237] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:49.643834 5960 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.643921 6096 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.644013 5561 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.644299 9805 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.644449 10517 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.644615 7549 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.645509 10517 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.645747 5430 consensus_queue.cc:237] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } }
I20240501 19:57:49.647486 6097 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.647553 5426 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.647801 5962 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.647863 10266 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.648133 7683 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 19:57:49.648321 5364 consensus_peers.cc:485] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa -> Peer 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499): Couldn't send request to peer 1865147cd5a346ad8549f18fa8cc26c3. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.648490 10266 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.648789 10266 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.649638 10485 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.649905 9805 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240501 19:57:49.650303 5629 consensus_peers.cc:485] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 064f698b376e407da78c7d439553896a. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.651798 5562 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.651796 5694 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.651911 5296 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.652174 10266 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.652490 10266 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.653401 5295 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.653874 10526 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.653991 10517 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.655047 10266 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.657227 5562 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.657757 5694 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } }
I20240501 19:57:49.658682 5149 catalog_manager.cc:5484] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 7 to 8, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New cstate: current_term: 1 leader_uuid: "9782508a47ba4403a39ca49fff7677fa" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:49.660173 10485 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.660514 6097 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.660558 5426 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.660859 5295 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.661374 9805 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.662796 7683 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.663051 5561 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.663522 5295 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.664609 7548 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.665299 9805 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.665808 5562 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.665673 5148 catalog_manager.cc:5484] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:49.666433 7417 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.667491 5827 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.668017 5962 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.668422 10540 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.668864 10540 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.668812 6097 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.669186 10517 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.672096 7549 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.672574 10517 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.674403 10519 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } }
W20240501 19:57:49.674662 5628 consensus_peers.cc:485] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae -> Peer 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469): Couldn't send request to peer 9782508a47ba4403a39ca49fff7677fa. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.674909 5295 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } }
I20240501 19:57:49.675704 5561 consensus_queue.cc:237] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } }
I20240501 19:57:49.678263 5829 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.678413 5430 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.678623 7683 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.679147 10352 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.679280 6097 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.679387 5962 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.679452 10352 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.679780 9958 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.680213 10374 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.680498 10374 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.681793 5695 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.682363 5151 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index 10 to 11, NON_VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } }
W20240501 19:57:49.682678 5499 consensus_peers.cc:485] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519): Couldn't send request to peer 4d4855c096ac4e1e936abf15834332ae. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.682888 5295 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.683244 9958 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.685911 6095 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } }
I20240501 19:57:49.685930 7417 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } }
I20240501 19:57:49.686515 7549 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } }
I20240501 19:57:49.686748 5296 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.691112 10563 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } }
I20240501 19:57:49.694546 7683 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } }
I20240501 19:57:49.694628 5962 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } }
I20240501 19:57:49.695070 6096 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.695511 5827 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } }
I20240501 19:57:49.698810 5562 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } }
I20240501 19:57:49.700039 5150 catalog_manager.cc:5484] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 7 to 8, NON_VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:49.700698 5829 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } }
I20240501 19:57:49.700825 5430 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } }
I20240501 19:57:49.702735 6097 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } }
I20240501 19:57:49.704806 5295 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } }
I20240501 19:57:49.709483 5960 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.716456 5695 consensus_queue.cc:237] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } }
I20240501 19:57:49.717298 5561 consensus_queue.cc:237] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: NON_VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: true } }
I20240501 19:57:49.718420 5295 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.719049 5430 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.719209 7683 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.719383 6096 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.719467 7417 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.719547 5960 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.719744 10374 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.719815 6097 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.719877 10485 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.720091 5962 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.720674 7549 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240501 19:57:49.720955 5496 consensus_peers.cc:485] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 -> Peer ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285): Couldn't send request to peer ad52038f1d38489ea464a8fbc72daac0. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.721166 5430 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.721349 5695 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.721410 5562 consensus_queue.cc:237] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:49.721544 10374 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.722141 7683 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.722438 10517 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20240501 19:57:49.722646 5629 consensus_peers.cc:485] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae -> Peer 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499): Couldn't send request to peer 1865147cd5a346ad8549f18fa8cc26c3. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.722689 9958 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.722815 10517 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.722970 9805 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.723311 9805 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.723520 6097 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.723520 9958 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.723624 9805 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.723637 10374 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.724550 7418 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.724851 5429 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.725513 7683 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240501 19:57:49.725771 5497 consensus_peers.cc:485] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c6ba23a7b4d24e0aa077e47798e890b8. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.725926 10352 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.726222 10352 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.726311 5960 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.726610 10352 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.726953 10352 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.727429 10352 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: NON_VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: true } } }
I20240501 19:57:49.728276 9958 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.728565 9958 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.729632 5695 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.729748 6096 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: NON_VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: true } } }
I20240501 19:57:49.730176 5694 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: NON_VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: true } } }
I20240501 19:57:49.731154 5962 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: NON_VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: true } } }
I20240501 19:57:49.731943 10519 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.733086 5154 catalog_manager.cc:5484] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 10 to 11, NON_VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: NON_VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: true } } }
I20240501 19:57:49.733232 5426 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: NON_VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: true } } }
I20240501 19:57:49.733846 6096 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.734818 5962 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.736557 5430 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.737615 10591 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.738919 5149 catalog_manager.cc:5484] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 10 to 11, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.741484 5295 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.742053 7549 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: NON_VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: true } } }
I20240501 19:57:49.742825 10565 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.743965 7418 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.744655 6097 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.745545 5961 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.747159 7682 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: NON_VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: true } } }
I20240501 19:57:49.749306 5151 catalog_manager.cc:5484] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 10 to 11, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:49.749387 5426 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.751237 7683 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } } }
I20240501 19:57:49.758658 7417 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.766846 5295 consensus_queue.cc:237] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
W20240501 19:57:49.769070 5231 consensus_peers.cc:485] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 -> Peer 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): Couldn't send request to peer 363f4ac4964f41a6ba0f98d1f487db83. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 913563ed5f904cc28ff15148d9a5e5a2. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.769178 5562 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.769615 5426 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.769699 5829 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.770002 6097 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.769986 10034 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.770107 5961 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.770380 10388 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.770718 10388 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.770846 10034 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.771176 10034 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.772322 7683 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:49.772665 10388 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:49.774410 10034 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.774574 5426 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.777055 5149 catalog_manager.cc:5484] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.777521 5829 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.779476 5694 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.781338 5562 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.781508 7684 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.782402 7683 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.783248 5961 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.790064 6097 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.811782 5829 consensus_queue.cc:237] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: true } }
I20240501 19:57:49.811895 5694 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:49.813967 5295 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.814123 7417 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.814404 10361 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.814551 10641 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.815186 7416 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
W20240501 19:57:49.815594 5629 consensus_peers.cc:485] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f0df1e5e85294f60b8de22053dd577f7. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.815616 5430 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240501 19:57:49.815850 5764 consensus_peers.cc:485] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 -> Peer 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085): Couldn't send request to peer 9544b0e566c54a62a6135f3bff47c4f2. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.815932 5562 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:49.815964 7549 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:49.816083 6097 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.816226 7548 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.816267 6096 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:49.816476 10641 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.816586 5694 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.816875 10519 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:49.817129 10519 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:49.817914 10641 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.820271 10361 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: true } } }
I20240501 19:57:49.820870 6096 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: true } } }
I20240501 19:57:49.820984 5694 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: true } } }
I20240501 19:57:49.821848 5295 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:49.821897 10642 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.816488 5829 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:49.823096 7418 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: true } } }
I20240501 19:57:49.823638 5149 catalog_manager.cc:5484] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 10 to 11, NON_VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) added. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240501 19:57:49.825134 5628 consensus_peers.cc:485] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae -> Peer 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469): Couldn't send request to peer 9782508a47ba4403a39ca49fff7677fa. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.825693 10649 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.826293 5430 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: true } } }
I20240501 19:57:49.826691 5296 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: true } } }
I20240501 19:57:49.826956 7548 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: true } } }
I20240501 19:57:49.827862 10519 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:49.829809 10520 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:49.830085 10520 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:49.830320 10539 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:49.833004 10519 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.835215 7548 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.835300 5561 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.836354 7418 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.836596 6096 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.837715 5296 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.838277 5149 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 11 to 12, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:49.841403 5829 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:49.853693 5694 consensus_queue.cc:237] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:49.855398 6096 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.855463 5296 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.855607 5561 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.855942 5961 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.855957 7548 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.856204 7683 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:49.856289 10539 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.856621 10539 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.856778 10540 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.856897 10539 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20240501 19:57:49.857128 5629 consensus_peers.cc:485] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae -> Peer 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): Couldn't send request to peer 363f4ac4964f41a6ba0f98d1f487db83. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bd8a8b5e92c74598a5399fe0ca8c048d. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.857144 10539 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20240501 19:57:49.858116 10517 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:49.860708 10539 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.861490 5296 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.861877 7683 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.864286 5961 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.864297 5145 catalog_manager.cc:5484] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 10 to 11, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:49.865957 7548 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.871306 6096 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.875942 5561 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } } }
I20240501 19:57:49.882953 5561 consensus_queue.cc:237] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:49.885093 6096 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:49.885176 5829 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:49.885224 5961 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:49.885569 10374 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:49.885879 10374 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20240501 19:57:49.885990 5497 consensus_peers.cc:485] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9ecbb3dec08047478ded9809f3a0b859. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.886195 10591 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:49.887993 5430 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:49.888015 7683 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:49.888411 10563 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:49.888559 10566 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20240501 19:57:49.888792 5499 consensus_peers.cc:485] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519): Couldn't send request to peer 4d4855c096ac4e1e936abf15834332ae. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:49.889338 10374 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.890511 5430 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.890698 7683 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.891237 6096 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.893163 5296 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:49.893568 5147 catalog_manager.cc:5484] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b reported cstate change: config changed from index 8 to 9, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.894721 5961 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.896646 5829 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:49.899121 10566 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:49.900080 5296 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:50.037200 10729 ts_tablet_manager.cc:902] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:50.037657 10729 tablet_copy_client.cc:308] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:50.038101 5715 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet 064f698b376e407da78c7d439553896a from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:34389)
I20240501 19:57:50.038203 5715 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet 064f698b376e407da78c7d439553896a from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:34389: session id = 16737daa1d234f35826143a1aa29dc0b-064f698b376e407da78c7d439553896a
I20240501 19:57:50.038928 5715 tablet_copy_source_session.cc:205] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:50.039748 10729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064f698b376e407da78c7d439553896a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:50.041671 10729 tablet_copy_client.cc:783] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:50.041851 10729 tablet_copy_client.cc:647] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:50.042927 10729 tablet_copy_client.cc:523] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:50.043913 10729 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:50.061179 10729 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:50.061442 10729 tablet_bootstrap.cc:492] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:50.061578 10729 ts_tablet_manager.cc:1367] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.005s
I20240501 19:57:50.061897 10729 raft_consensus.cc:348] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:50.062004 10729 raft_consensus.cc:729] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:50.062083 10729 consensus_queue.cc:260] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:50.062371 10729 ts_tablet_manager.cc:1398] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 19:57:50.062845 5715 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-064f698b376e407da78c7d439553896a received from {username='slave'} at 127.20.210.10:34389
I20240501 19:57:50.062929 5715 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-064f698b376e407da78c7d439553896a on tablet 064f698b376e407da78c7d439553896a with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:50.100662 10740 ts_tablet_manager.cc:902] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:50.101217 10740 tablet_copy_client.cc:308] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:50.102908 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet c6ba23a7b4d24e0aa077e47798e890b8 from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:42563)
I20240501 19:57:50.103029 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet c6ba23a7b4d24e0aa077e47798e890b8 from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:42563: session id = 143c54da1456472dadbf1d5d65e0a6f6-c6ba23a7b4d24e0aa077e47798e890b8
I20240501 19:57:50.103873 5583 tablet_copy_source_session.cc:205] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:50.104837 10740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ba23a7b4d24e0aa077e47798e890b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:50.107219 10740 tablet_copy_client.cc:783] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:50.107371 10740 tablet_copy_client.cc:647] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:50.108407 10740 tablet_copy_client.cc:523] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:50.109957 10740 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:50.135731 10740 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:50.136004 10740 tablet_bootstrap.cc:492] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:50.136170 10740 ts_tablet_manager.cc:1367] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.026s user 0.025s sys 0.000s
I20240501 19:57:50.136583 10740 raft_consensus.cc:348] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:50.136723 10740 raft_consensus.cc:729] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:50.136827 10740 consensus_queue.cc:260] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:50.137188 10740 ts_tablet_manager.cc:1398] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 19:57:50.137766 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-c6ba23a7b4d24e0aa077e47798e890b8 received from {username='slave'} at 127.20.210.9:42563
I20240501 19:57:50.137845 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-c6ba23a7b4d24e0aa077e47798e890b8 on tablet c6ba23a7b4d24e0aa077e47798e890b8 with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:50.188987 10797 ts_tablet_manager.cc:902] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: Initiating tablet copy from peer 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
I20240501 19:57:50.194061 10797 tablet_copy_client.cc:308] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.2:35469
I20240501 19:57:50.195140 5450 tablet_copy_service.cc:140] P 9782508a47ba4403a39ca49fff7677fa: Received BeginTabletCopySession request for tablet 0681575c2bb142e0915be3dc75858f87 from peer 1865147cd5a346ad8549f18fa8cc26c3 ({username='slave'} at 127.20.210.5:60621)
I20240501 19:57:50.195283 5450 tablet_copy_service.cc:161] P 9782508a47ba4403a39ca49fff7677fa: Beginning new tablet copy session on tablet 0681575c2bb142e0915be3dc75858f87 from peer 1865147cd5a346ad8549f18fa8cc26c3 at {username='slave'} at 127.20.210.5:60621: session id = 1865147cd5a346ad8549f18fa8cc26c3-0681575c2bb142e0915be3dc75858f87
I20240501 19:57:50.196184 5450 tablet_copy_source_session.cc:205] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:50.197553 10797 ts_tablet_manager.cc:1875] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 19:57:50.200299 10797 ts_tablet_manager.cc:1888] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 19:57:50.200445 10797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0681575c2bb142e0915be3dc75858f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:50.201344 10797 tablet_copy_client.cc:783] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:50.201478 10797 tablet_copy_client.cc:647] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:50.202868 10797 tablet_copy_client.cc:523] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:50.203430 10807 ts_tablet_manager.cc:902] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:50.204161 10807 tablet_copy_client.cc:308] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:50.204308 10797 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:50.204644 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet d3937a2d841440d793ba2637f1af060f from peer ad52038f1d38489ea464a8fbc72daac0 ({username='slave'} at 127.20.210.1:58985)
I20240501 19:57:50.204744 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet d3937a2d841440d793ba2637f1af060f from peer ad52038f1d38489ea464a8fbc72daac0 at {username='slave'} at 127.20.210.1:58985: session id = ad52038f1d38489ea464a8fbc72daac0-d3937a2d841440d793ba2637f1af060f
I20240501 19:57:50.205714 5583 tablet_copy_source_session.cc:205] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:50.206642 10807 ts_tablet_manager.cc:1875] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 19:57:50.208808 10807 ts_tablet_manager.cc:1888] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240501 19:57:50.208964 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3937a2d841440d793ba2637f1af060f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:50.209853 10813 ts_tablet_manager.cc:902] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:50.210330 10813 tablet_copy_client.cc:308] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:50.210700 10807 tablet_copy_client.cc:783] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:50.210786 5715 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet cb5540ce8d3343a68a8bf7d8fff00589 from peer 1865147cd5a346ad8549f18fa8cc26c3 ({username='slave'} at 127.20.210.5:57597)
I20240501 19:57:50.210858 10807 tablet_copy_client.cc:647] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:50.210882 5715 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet cb5540ce8d3343a68a8bf7d8fff00589 from peer 1865147cd5a346ad8549f18fa8cc26c3 at {username='slave'} at 127.20.210.5:57597: session id = 1865147cd5a346ad8549f18fa8cc26c3-cb5540ce8d3343a68a8bf7d8fff00589
I20240501 19:57:50.211757 5715 tablet_copy_source_session.cc:205] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:50.212108 10807 tablet_copy_client.cc:523] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:50.212461 10813 ts_tablet_manager.cc:1875] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 19:57:50.213232 10807 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: Bootstrap starting.
I20240501 19:57:50.214367 10813 ts_tablet_manager.cc:1888] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 19:57:50.214560 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5540ce8d3343a68a8bf7d8fff00589. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:50.215692 10813 tablet_copy_client.cc:783] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:50.215873 10813 tablet_copy_client.cc:647] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:50.219285 10813 tablet_copy_client.cc:523] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:50.220382 10813 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap starting.
I20240501 19:57:50.231860 10797 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:50.232136 10797 tablet_bootstrap.cc:492] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap complete.
I20240501 19:57:50.232313 10797 ts_tablet_manager.cc:1367] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.028s user 0.021s sys 0.005s
I20240501 19:57:50.232723 10797 raft_consensus.cc:348] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } }
I20240501 19:57:50.232858 10797 raft_consensus.cc:729] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: LEARNER
I20240501 19:57:50.232969 10797 consensus_queue.cc:260] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } }
I20240501 19:57:50.233340 10797 ts_tablet_manager.cc:1398] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 19:57:50.234090 5450 tablet_copy_service.cc:342] P 9782508a47ba4403a39ca49fff7677fa: Request end of tablet copy session 1865147cd5a346ad8549f18fa8cc26c3-0681575c2bb142e0915be3dc75858f87 received from {username='slave'} at 127.20.210.5:60621
I20240501 19:57:50.234174 5450 tablet_copy_service.cc:433] P 9782508a47ba4403a39ca49fff7677fa: ending tablet copy session 1865147cd5a346ad8549f18fa8cc26c3-0681575c2bb142e0915be3dc75858f87 on tablet 0681575c2bb142e0915be3dc75858f87 with peer 1865147cd5a346ad8549f18fa8cc26c3
I20240501 19:57:50.236377 10807 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:50.236677 10807 tablet_bootstrap.cc:492] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: Bootstrap complete.
I20240501 19:57:50.236855 10807 ts_tablet_manager.cc:1367] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: Time spent bootstrapping tablet: real 0.024s user 0.023s sys 0.000s
I20240501 19:57:50.237237 10807 raft_consensus.cc:348] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: NON_VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: true } }
I20240501 19:57:50.237340 10807 raft_consensus.cc:729] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ad52038f1d38489ea464a8fbc72daac0, State: Initialized, Role: LEARNER
I20240501 19:57:50.237421 10807 consensus_queue.cc:260] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: NON_VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: true } }
I20240501 19:57:50.237687 10807 ts_tablet_manager.cc:1398] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:50.238520 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session ad52038f1d38489ea464a8fbc72daac0-d3937a2d841440d793ba2637f1af060f received from {username='slave'} at 127.20.210.1:58985
I20240501 19:57:50.238597 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session ad52038f1d38489ea464a8fbc72daac0-d3937a2d841440d793ba2637f1af060f on tablet d3937a2d841440d793ba2637f1af060f with peer ad52038f1d38489ea464a8fbc72daac0
I20240501 19:57:50.240455 10821 ts_tablet_manager.cc:902] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:50.241103 10821 tablet_copy_client.cc:308] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:50.241591 5715 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet f0df1e5e85294f60b8de22053dd577f7 from peer 9782508a47ba4403a39ca49fff7677fa ({username='slave'} at 127.20.210.2:56729)
I20240501 19:57:50.241685 5715 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet f0df1e5e85294f60b8de22053dd577f7 from peer 9782508a47ba4403a39ca49fff7677fa at {username='slave'} at 127.20.210.2:56729: session id = 9782508a47ba4403a39ca49fff7677fa-f0df1e5e85294f60b8de22053dd577f7
I20240501 19:57:50.242484 5715 tablet_copy_source_session.cc:205] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:50.243386 10821 ts_tablet_manager.cc:1875] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 19:57:50.245328 10821 ts_tablet_manager.cc:1888] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240501 19:57:50.245472 10821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0df1e5e85294f60b8de22053dd577f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:50.246348 10821 tablet_copy_client.cc:783] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:50.246505 10821 tablet_copy_client.cc:647] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:50.247598 10813 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:50.247758 10821 tablet_copy_client.cc:523] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:50.247844 10813 tablet_bootstrap.cc:492] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: Bootstrap complete.
I20240501 19:57:50.247987 10813 ts_tablet_manager.cc:1367] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent bootstrapping tablet: real 0.028s user 0.022s sys 0.002s
I20240501 19:57:50.248313 10813 raft_consensus.cc:348] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } }
I20240501 19:57:50.248441 10813 raft_consensus.cc:729] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1865147cd5a346ad8549f18fa8cc26c3, State: Initialized, Role: LEARNER
I20240501 19:57:50.248529 10813 consensus_queue.cc:260] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: NON_VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: true } }
I20240501 19:57:50.248790 10813 ts_tablet_manager.cc:1398] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:50.249222 5715 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 1865147cd5a346ad8549f18fa8cc26c3-cb5540ce8d3343a68a8bf7d8fff00589 received from {username='slave'} at 127.20.210.5:57597
I20240501 19:57:50.249261 10821 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap starting.
I20240501 19:57:50.249296 5715 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 1865147cd5a346ad8549f18fa8cc26c3-cb5540ce8d3343a68a8bf7d8fff00589 on tablet cb5540ce8d3343a68a8bf7d8fff00589 with peer 1865147cd5a346ad8549f18fa8cc26c3
I20240501 19:57:50.268981 10821 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:50.269178 10821 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: Bootstrap complete.
I20240501 19:57:50.269305 10821 ts_tablet_manager.cc:1367] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: Time spent bootstrapping tablet: real 0.020s user 0.017s sys 0.000s
I20240501 19:57:50.269613 10821 raft_consensus.cc:348] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:50.269708 10821 raft_consensus.cc:729] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9782508a47ba4403a39ca49fff7677fa, State: Initialized, Role: LEARNER
I20240501 19:57:50.269791 10821 consensus_queue.cc:260] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:50.269990 10821 ts_tablet_manager.cc:1398] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:50.270383 5715 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 9782508a47ba4403a39ca49fff7677fa-f0df1e5e85294f60b8de22053dd577f7 received from {username='slave'} at 127.20.210.2:56729
I20240501 19:57:50.270449 5715 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 9782508a47ba4403a39ca49fff7677fa-f0df1e5e85294f60b8de22053dd577f7 on tablet f0df1e5e85294f60b8de22053dd577f7 with peer 9782508a47ba4403a39ca49fff7677fa
I20240501 19:57:50.277773 10827 ts_tablet_manager.cc:902] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:50.278317 10827 tablet_copy_client.cc:308] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:50.278829 5715 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet bd8a8b5e92c74598a5399fe0ca8c048d from peer 363f4ac4964f41a6ba0f98d1f487db83 ({username='slave'} at 127.20.210.8:53017)
I20240501 19:57:50.278915 5715 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet bd8a8b5e92c74598a5399fe0ca8c048d from peer 363f4ac4964f41a6ba0f98d1f487db83 at {username='slave'} at 127.20.210.8:53017: session id = 363f4ac4964f41a6ba0f98d1f487db83-bd8a8b5e92c74598a5399fe0ca8c048d
I20240501 19:57:50.279536 5715 tablet_copy_source_session.cc:205] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:50.280403 10827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd8a8b5e92c74598a5399fe0ca8c048d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:50.283552 10827 tablet_copy_client.cc:783] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:50.283797 10827 tablet_copy_client.cc:647] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:50.291368 10827 tablet_copy_client.cc:523] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:50.292996 10827 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap starting.
I20240501 19:57:50.306535 10833 ts_tablet_manager.cc:902] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: Initiating tablet copy from peer 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5:39499)
I20240501 19:57:50.307355 10833 tablet_copy_client.cc:308] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.5:39499
I20240501 19:57:50.307924 5849 tablet_copy_service.cc:140] P 1865147cd5a346ad8549f18fa8cc26c3: Received BeginTabletCopySession request for tablet d8a84349ba37496289351ac689f77eab from peer 9544b0e566c54a62a6135f3bff47c4f2 ({username='slave'} at 127.20.210.6:54183)
I20240501 19:57:50.308051 5849 tablet_copy_service.cc:161] P 1865147cd5a346ad8549f18fa8cc26c3: Beginning new tablet copy session on tablet d8a84349ba37496289351ac689f77eab from peer 9544b0e566c54a62a6135f3bff47c4f2 at {username='slave'} at 127.20.210.6:54183: session id = 9544b0e566c54a62a6135f3bff47c4f2-d8a84349ba37496289351ac689f77eab
I20240501 19:57:50.308852 5849 tablet_copy_source_session.cc:205] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:50.309813 10833 ts_tablet_manager.cc:1875] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 19:57:50.311998 10833 ts_tablet_manager.cc:1888] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 19:57:50.312176 10833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8a84349ba37496289351ac689f77eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:50.313400 10833 tablet_copy_client.cc:783] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:50.313553 10833 tablet_copy_client.cc:647] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:50.314908 10833 tablet_copy_client.cc:523] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:50.316815 10833 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap starting.
I20240501 19:57:50.329299 10827 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:50.329560 10827 tablet_bootstrap.cc:492] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap complete.
I20240501 19:57:50.329730 10827 ts_tablet_manager.cc:1367] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83: Time spent bootstrapping tablet: real 0.037s user 0.024s sys 0.005s
I20240501 19:57:50.330209 10827 raft_consensus.cc:348] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:50.330371 10827 raft_consensus.cc:729] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 363f4ac4964f41a6ba0f98d1f487db83, State: Initialized, Role: LEARNER
I20240501 19:57:50.330467 10827 consensus_queue.cc:260] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:50.330737 10827 ts_tablet_manager.cc:1398] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240501 19:57:50.331148 5715 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-bd8a8b5e92c74598a5399fe0ca8c048d received from {username='slave'} at 127.20.210.8:53017
I20240501 19:57:50.331228 5715 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-bd8a8b5e92c74598a5399fe0ca8c048d on tablet bd8a8b5e92c74598a5399fe0ca8c048d with peer 363f4ac4964f41a6ba0f98d1f487db83
I20240501 19:57:50.344378 10833 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:50.344619 10833 tablet_bootstrap.cc:492] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: Bootstrap complete.
I20240501 19:57:50.344767 10833 ts_tablet_manager.cc:1367] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: Time spent bootstrapping tablet: real 0.028s user 0.022s sys 0.004s
I20240501 19:57:50.345149 10833 raft_consensus.cc:348] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: true } }
I20240501 19:57:50.345249 10833 raft_consensus.cc:729] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9544b0e566c54a62a6135f3bff47c4f2, State: Initialized, Role: LEARNER
I20240501 19:57:50.345434 10833 consensus_queue.cc:260] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: NON_VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: true } }
I20240501 19:57:50.345728 10833 ts_tablet_manager.cc:1398] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240501 19:57:50.346225 5849 tablet_copy_service.cc:342] P 1865147cd5a346ad8549f18fa8cc26c3: Request end of tablet copy session 9544b0e566c54a62a6135f3bff47c4f2-d8a84349ba37496289351ac689f77eab received from {username='slave'} at 127.20.210.6:54183
I20240501 19:57:50.346320 5849 tablet_copy_service.cc:433] P 1865147cd5a346ad8549f18fa8cc26c3: ending tablet copy session 9544b0e566c54a62a6135f3bff47c4f2-d8a84349ba37496289351ac689f77eab on tablet d8a84349ba37496289351ac689f77eab with peer 9544b0e566c54a62a6135f3bff47c4f2
I20240501 19:57:50.347455 10827 ts_tablet_manager.cc:902] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83: Initiating tablet copy from peer ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1:46285)
I20240501 19:57:50.347790 10827 tablet_copy_client.cc:308] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.1:46285
I20240501 19:57:50.348361 5317 tablet_copy_service.cc:140] P ad52038f1d38489ea464a8fbc72daac0: Received BeginTabletCopySession request for tablet 913563ed5f904cc28ff15148d9a5e5a2 from peer 363f4ac4964f41a6ba0f98d1f487db83 ({username='slave'} at 127.20.210.8:58425)
I20240501 19:57:50.348461 5317 tablet_copy_service.cc:161] P ad52038f1d38489ea464a8fbc72daac0: Beginning new tablet copy session on tablet 913563ed5f904cc28ff15148d9a5e5a2 from peer 363f4ac4964f41a6ba0f98d1f487db83 at {username='slave'} at 127.20.210.8:58425: session id = 363f4ac4964f41a6ba0f98d1f487db83-913563ed5f904cc28ff15148d9a5e5a2
I20240501 19:57:50.349251 5317 tablet_copy_source_session.cc:205] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:50.350185 10827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 913563ed5f904cc28ff15148d9a5e5a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:50.351809 10827 tablet_copy_client.cc:783] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:50.351951 10827 tablet_copy_client.cc:647] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:50.352957 10827 tablet_copy_client.cc:523] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:50.353752 10827 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap starting.
I20240501 19:57:50.364148 10869 ts_tablet_manager.cc:902] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:50.364886 10869 tablet_copy_client.cc:308] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:50.365468 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet 9ecbb3dec08047478ded9809f3a0b859 from peer 4d4855c096ac4e1e936abf15834332ae ({username='slave'} at 127.20.210.4:42181)
I20240501 19:57:50.365590 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet 9ecbb3dec08047478ded9809f3a0b859 from peer 4d4855c096ac4e1e936abf15834332ae at {username='slave'} at 127.20.210.4:42181: session id = 4d4855c096ac4e1e936abf15834332ae-9ecbb3dec08047478ded9809f3a0b859
I20240501 19:57:50.366482 5583 tablet_copy_source_session.cc:205] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:50.367414 10869 ts_tablet_manager.cc:1875] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240501 19:57:50.369280 10869 ts_tablet_manager.cc:1888] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240501 19:57:50.369422 10869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ecbb3dec08047478ded9809f3a0b859. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:50.370213 10869 tablet_copy_client.cc:783] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:50.370765 10869 tablet_copy_client.cc:647] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:50.373029 10827 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:50.373273 10827 tablet_bootstrap.cc:492] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83: Bootstrap complete.
I20240501 19:57:50.373405 10827 ts_tablet_manager.cc:1367] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent bootstrapping tablet: real 0.020s user 0.013s sys 0.004s
I20240501 19:57:50.373456 10869 tablet_copy_client.cc:523] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:50.373749 10827 raft_consensus.cc:348] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:50.373893 10827 raft_consensus.cc:729] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 363f4ac4964f41a6ba0f98d1f487db83, State: Initialized, Role: LEARNER
I20240501 19:57:50.373991 10827 consensus_queue.cc:260] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: NON_VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: true } }
I20240501 19:57:50.374315 10827 ts_tablet_manager.cc:1398] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 19:57:50.374763 10869 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap starting.
I20240501 19:57:50.378573 5317 tablet_copy_service.cc:342] P ad52038f1d38489ea464a8fbc72daac0: Request end of tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-913563ed5f904cc28ff15148d9a5e5a2 received from {username='slave'} at 127.20.210.8:58425
I20240501 19:57:50.378654 5317 tablet_copy_service.cc:433] P ad52038f1d38489ea464a8fbc72daac0: ending tablet copy session 363f4ac4964f41a6ba0f98d1f487db83-913563ed5f904cc28ff15148d9a5e5a2 on tablet 913563ed5f904cc28ff15148d9a5e5a2 with peer 363f4ac4964f41a6ba0f98d1f487db83
I20240501 19:57:50.387681 10729 ts_tablet_manager.cc:902] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b: Initiating tablet copy from peer 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519)
I20240501 19:57:50.388092 10729 tablet_copy_client.cc:308] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.4:46519
I20240501 19:57:50.388633 5715 tablet_copy_service.cc:140] P 4d4855c096ac4e1e936abf15834332ae: Received BeginTabletCopySession request for tablet f0df1e5e85294f60b8de22053dd577f7 from peer 16737daa1d234f35826143a1aa29dc0b ({username='slave'} at 127.20.210.10:34389)
I20240501 19:57:50.388720 5715 tablet_copy_service.cc:161] P 4d4855c096ac4e1e936abf15834332ae: Beginning new tablet copy session on tablet f0df1e5e85294f60b8de22053dd577f7 from peer 16737daa1d234f35826143a1aa29dc0b at {username='slave'} at 127.20.210.10:34389: session id = 16737daa1d234f35826143a1aa29dc0b-f0df1e5e85294f60b8de22053dd577f7
I20240501 19:57:50.389683 5715 tablet_copy_source_session.cc:205] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:50.390507 10729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0df1e5e85294f60b8de22053dd577f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:50.392685 10729 tablet_copy_client.cc:783] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:50.392807 10729 tablet_copy_client.cc:647] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:50.394665 10729 tablet_copy_client.cc:523] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:50.395573 10729 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap starting.
I20240501 19:57:50.401773 10869 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:50.402060 10869 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: Bootstrap complete.
I20240501 19:57:50.402212 10869 ts_tablet_manager.cc:1367] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: Time spent bootstrapping tablet: real 0.027s user 0.019s sys 0.002s
I20240501 19:57:50.402706 10869 raft_consensus.cc:348] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:50.402892 10869 raft_consensus.cc:729] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4d4855c096ac4e1e936abf15834332ae, State: Initialized, Role: LEARNER
I20240501 19:57:50.402983 10869 consensus_queue.cc:260] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:50.403318 10869 ts_tablet_manager.cc:1398] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240501 19:57:50.403955 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session 4d4855c096ac4e1e936abf15834332ae-9ecbb3dec08047478ded9809f3a0b859 received from {username='slave'} at 127.20.210.4:42181
I20240501 19:57:50.404048 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session 4d4855c096ac4e1e936abf15834332ae-9ecbb3dec08047478ded9809f3a0b859 on tablet 9ecbb3dec08047478ded9809f3a0b859 with peer 4d4855c096ac4e1e936abf15834332ae
I20240501 19:57:50.420958 10729 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:50.421167 10729 tablet_bootstrap.cc:492] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b: Bootstrap complete.
I20240501 19:57:50.421320 10729 ts_tablet_manager.cc:1367] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b: Time spent bootstrapping tablet: real 0.026s user 0.024s sys 0.000s
I20240501 19:57:50.421659 10729 raft_consensus.cc:348] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:50.421774 10729 raft_consensus.cc:729] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 16737daa1d234f35826143a1aa29dc0b, State: Initialized, Role: LEARNER
I20240501 19:57:50.421886 10729 consensus_queue.cc:260] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:50.422111 10729 ts_tablet_manager.cc:1398] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240501 19:57:50.422506 5715 tablet_copy_service.cc:342] P 4d4855c096ac4e1e936abf15834332ae: Request end of tablet copy session 16737daa1d234f35826143a1aa29dc0b-f0df1e5e85294f60b8de22053dd577f7 received from {username='slave'} at 127.20.210.10:34389
I20240501 19:57:50.422593 5715 tablet_copy_service.cc:433] P 4d4855c096ac4e1e936abf15834332ae: ending tablet copy session 16737daa1d234f35826143a1aa29dc0b-f0df1e5e85294f60b8de22053dd577f7 on tablet f0df1e5e85294f60b8de22053dd577f7 with peer 16737daa1d234f35826143a1aa29dc0b
I20240501 19:57:50.431958 7683 raft_consensus.cc:1203] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:50.501138 10740 ts_tablet_manager.cc:902] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275)
I20240501 19:57:50.501600 10740 tablet_copy_client.cc:308] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Beginning tablet copy session from remote peer at address 127.20.210.3:44275
I20240501 19:57:50.502058 5583 tablet_copy_service.cc:140] P f5287d5ae4d84bd692b0f7985cd94171: Received BeginTabletCopySession request for tablet 9ecbb3dec08047478ded9809f3a0b859 from peer 143c54da1456472dadbf1d5d65e0a6f6 ({username='slave'} at 127.20.210.9:42563)
I20240501 19:57:50.502148 5583 tablet_copy_service.cc:161] P f5287d5ae4d84bd692b0f7985cd94171: Beginning new tablet copy session on tablet 9ecbb3dec08047478ded9809f3a0b859 from peer 143c54da1456472dadbf1d5d65e0a6f6 at {username='slave'} at 127.20.210.9:42563: session id = 143c54da1456472dadbf1d5d65e0a6f6-9ecbb3dec08047478ded9809f3a0b859
I20240501 19:57:50.502856 5583 tablet_copy_source_session.cc:205] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171: Tablet Copy: opened 0 blocks and 1 log segments
I20240501 19:57:50.503605 10740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ecbb3dec08047478ded9809f3a0b859. 1 dirs total, 0 dirs full, 0 dirs failed
I20240501 19:57:50.505290 10740 tablet_copy_client.cc:783] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 0 data blocks...
I20240501 19:57:50.505441 10740 tablet_copy_client.cc:647] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Starting download of 1 WAL segments...
I20240501 19:57:50.506469 10740 tablet_copy_client.cc:523] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240501 19:57:50.507337 10740 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap starting.
I20240501 19:57:50.525321 10740 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240501 19:57:50.525534 10740 tablet_bootstrap.cc:492] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6: Bootstrap complete.
I20240501 19:57:50.525676 10740 ts_tablet_manager.cc:1367] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.004s
I20240501 19:57:50.526031 10740 raft_consensus.cc:348] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:50.526132 10740 raft_consensus.cc:729] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 143c54da1456472dadbf1d5d65e0a6f6, State: Initialized, Role: LEARNER
I20240501 19:57:50.526229 10740 consensus_queue.cc:260] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: NON_VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: true } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:50.526538 10740 ts_tablet_manager.cc:1398] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240501 19:57:50.527228 5583 tablet_copy_service.cc:342] P f5287d5ae4d84bd692b0f7985cd94171: Request end of tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-9ecbb3dec08047478ded9809f3a0b859 received from {username='slave'} at 127.20.210.9:42563
I20240501 19:57:50.527302 5583 tablet_copy_service.cc:433] P f5287d5ae4d84bd692b0f7985cd94171: ending tablet copy session 143c54da1456472dadbf1d5d65e0a6f6-9ecbb3dec08047478ded9809f3a0b859 on tablet 9ecbb3dec08047478ded9809f3a0b859 with peer 143c54da1456472dadbf1d5d65e0a6f6
I20240501 19:57:50.554813 7548 raft_consensus.cc:1203] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 19:57:50.628232 5296 raft_consensus.cc:1203] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 19:57:50.657574 7418 raft_consensus.cc:1203] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 19:57:50.730211 5829 raft_consensus.cc:1203] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 19:57:50.740218 5829 raft_consensus.cc:1203] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:50.790781 5430 raft_consensus.cc:1203] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEARNER]: Deduplicated request from leader. Original: 1.11->[1.12-1.12] Dedup: 1.12->[]
I20240501 19:57:50.811610 5695 raft_consensus.cc:1203] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEARNER]: Deduplicated request from leader. Original: 1.8->[1.9-1.9] Dedup: 1.9->[]
I20240501 19:57:50.841684 10520 raft_consensus.cc:1050] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:50.842161 10520 consensus_queue.cc:237] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:50.844108 5561 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:50.844123 7548 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:50.844226 7683 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:50.844264 5962 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:50.844568 10520 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:50.844643 6096 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:50.844952 10916 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:50.845006 5430 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:50.845423 10520 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:50.846041 5296 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:50.846077 10520 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:50.846299 10916 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:50.846422 10520 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:50.846542 11017 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:50.849011 11018 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.849489 5561 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.850272 6096 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.851500 7683 raft_consensus.cc:1203] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.11->[1.12-1.12] Dedup: 1.12->[]
I20240501 19:57:50.851420 5962 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.851907 7684 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.853159 7418 raft_consensus.cc:1203] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240501 19:57:50.853845 7548 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.854288 5149 catalog_manager.cc:5484] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.854413 5430 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.855397 5296 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.865622 5695 consensus_queue.cc:237] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:50.867288 7684 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:50.867480 5430 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:50.867545 5296 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:50.867832 11016 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:50.867937 5962 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:50.867980 11017 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:50.868480 5561 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:50.868880 5961 raft_consensus.cc:1203] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240501 19:57:50.868988 6096 raft_consensus.cc:1261] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:50.870114 11016 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:50.870285 10916 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:50.870816 10520 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:50.871212 10916 consensus_queue.cc:1035] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:50.873036 10916 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.875012 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 064f698b376e407da78c7d439553896a with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:50.875371 5145 catalog_manager.cc:5484] T 064f698b376e407da78c7d439553896a P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 9 to 10, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:50.875360 6096 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.875545 5961 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.876899 5561 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.877718 5296 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.878540 7684 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.879210 5430 raft_consensus.cc:2932] T 064f698b376e407da78c7d439553896a P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:50.889612 7530 tablet_service.cc:1514] Processing DeleteTablet for tablet 064f698b376e407da78c7d439553896a with delete_type TABLET_DATA_TOMBSTONED (TS 143c54da1456472dadbf1d5d65e0a6f6 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:36080
I20240501 19:57:50.890062 11050 tablet_replica.cc:330] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: stopping tablet replica
I20240501 19:57:50.890421 11050 raft_consensus.cc:2229] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:50.890741 11050 raft_consensus.cc:2258] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:50.892302 11050 ts_tablet_manager.cc:1875] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:50.895439 11050 ts_tablet_manager.cc:1888] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:50.895524 11050 log.cc:1196] T 064f698b376e407da78c7d439553896a P 143c54da1456472dadbf1d5d65e0a6f6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal/wals/064f698b376e407da78c7d439553896a
I20240501 19:57:50.896085 5139 catalog_manager.cc:4830] TS 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): tablet 064f698b376e407da78c7d439553896a (table rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]) successfully deleted
I20240501 19:57:51.008262 10563 raft_consensus.cc:1050] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER ad52038f1d38489ea464a8fbc72daac0 to VOTER
I20240501 19:57:51.008767 10563 consensus_queue.cc:237] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } }
I20240501 19:57:51.010972 5694 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.011329 7684 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.011704 5961 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.011755 11108 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.012056 6096 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.012814 5296 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.012938 7548 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.013204 11110 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.013373 5430 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.013424 10591 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.013566 11109 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.013768 11111 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.013901 10566 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.014053 11108 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.017582 11110 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.018011 5961 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.019351 6096 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.020612 7684 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.020656 5430 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.021106 7548 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.021920 5694 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.022472 5147 catalog_manager.cc:5484] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 11 to 12, ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.024674 5296 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ad52038f1d38489ea464a8fbc72daac0 (127.20.210.1) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { replace: true } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.030259 5562 consensus_queue.cc:237] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } }
I20240501 19:57:51.031940 5296 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.032021 7548 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.032354 10566 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.032497 11107 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.032644 5694 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.033048 11107 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.033118 7684 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.033463 11108 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.034070 6096 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.034508 11108 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.034747 5961 raft_consensus.cc:1261] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.035305 10566 consensus_queue.cc:1035] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.035681 10563 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.036149 5296 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.036367 7684 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.037180 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d3937a2d841440d793ba2637f1af060f with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:51.038888 5145 catalog_manager.cc:5484] T d3937a2d841440d793ba2637f1af060f P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index 12 to 13, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.038954 7548 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.041041 5694 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.041396 5408 tablet_service.cc:1514] Processing DeleteTablet for tablet d3937a2d841440d793ba2637f1af060f with delete_type TABLET_DATA_TOMBSTONED (TS 9782508a47ba4403a39ca49fff7677fa not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:43934
I20240501 19:57:51.042896 6096 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.044224 11128 tablet_replica.cc:330] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa: stopping tablet replica
I20240501 19:57:51.044433 11128 raft_consensus.cc:2229] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:51.044768 11128 raft_consensus.cc:2258] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:51.045526 5961 raft_consensus.cc:2932] T d3937a2d841440d793ba2637f1af060f P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } attrs { promote: false } } }
I20240501 19:57:51.046054 11128 ts_tablet_manager.cc:1875] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:51.048409 11128 ts_tablet_manager.cc:1888] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 19:57:51.048527 11128 log.cc:1196] T d3937a2d841440d793ba2637f1af060f P 9782508a47ba4403a39ca49fff7677fa: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-1/wal/wals/d3937a2d841440d793ba2637f1af060f
I20240501 19:57:51.049327 5138 catalog_manager.cc:4830] TS 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469): tablet d3937a2d841440d793ba2637f1af060f (table rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]) successfully deleted
I20240501 19:57:51.124668 7548 raft_consensus.cc:1203] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.8->[1.9-1.9] Dedup: 1.9->[]
I20240501 19:57:51.169430 10566 raft_consensus.cc:1050] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:51.169813 10520 raft_consensus.cc:1050] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 to VOTER
I20240501 19:57:51.170002 10566 consensus_queue.cc:237] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:51.170300 10520 consensus_queue.cc:237] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:51.172102 7684 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.172389 5430 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.172577 7418 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.172782 6096 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.172870 5962 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.172886 6097 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.173126 5961 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.173175 11168 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.173362 10591 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.173544 11107 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.173702 11110 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.174026 11110 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.174865 11108 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.174960 5562 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.175179 11167 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.175376 11016 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.175568 5296 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.175896 11167 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.176486 7418 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.176618 5694 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.176903 7684 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.177577 7548 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.177577 7549 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.179047 11016 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.179191 10916 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.179422 11016 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.179999 11111 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.180142 10566 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.181840 10916 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.182515 10591 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.182958 7684 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.185338 5562 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.185571 5296 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.185823 6096 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.187204 5962 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.187258 6097 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.187352 5430 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.189167 5147 catalog_manager.cc:5484] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 11 to 12, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:51.189728 5694 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.190999 7549 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.192306 7418 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.194515 7416 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.189178 5151 catalog_manager.cc:5484] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 11 to 12, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:51.196606 7684 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.196946 5695 consensus_queue.cc:237] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:51.199211 5562 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.199319 6097 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.199626 5961 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.199707 10916 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.199947 5962 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.200619 7684 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.200884 5296 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.201318 11168 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.201813 7416 raft_consensus.cc:1261] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.203006 10916 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.203141 10520 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.203603 10916 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.204010 10520 consensus_queue.cc:1035] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.205372 10916 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.205766 6097 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.206077 7416 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.207643 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bd8a8b5e92c74598a5399fe0ca8c048d with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:51.208026 5562 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.208093 5296 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.209570 7684 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.210402 5147 catalog_manager.cc:5484] T bd8a8b5e92c74598a5399fe0ca8c048d P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 12 to 13, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:51.210978 5962 raft_consensus.cc:2932] T bd8a8b5e92c74598a5399fe0ca8c048d P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.213049 7548 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.216161 7530 tablet_service.cc:1514] Processing DeleteTablet for tablet bd8a8b5e92c74598a5399fe0ca8c048d with delete_type TABLET_DATA_TOMBSTONED (TS 143c54da1456472dadbf1d5d65e0a6f6 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:36080
I20240501 19:57:51.216358 11050 tablet_replica.cc:330] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: stopping tablet replica
I20240501 19:57:51.216495 11050 raft_consensus.cc:2229] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:51.216682 11050 raft_consensus.cc:2258] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:51.217281 11050 ts_tablet_manager.cc:1875] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:51.217828 5562 consensus_queue.cc:237] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:51.219110 11050 ts_tablet_manager.cc:1888] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 19:57:51.219183 11050 log.cc:1196] T bd8a8b5e92c74598a5399fe0ca8c048d P 143c54da1456472dadbf1d5d65e0a6f6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal/wals/bd8a8b5e92c74598a5399fe0ca8c048d
I20240501 19:57:51.219744 5139 catalog_manager.cc:4830] TS 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): tablet bd8a8b5e92c74598a5399fe0ca8c048d (table rebalance_test_table_0 [id=64fd4c18503b42e1ba26bfc1dd3541f0]) successfully deleted
I20240501 19:57:51.220064 5962 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.220427 5694 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.220525 11108 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.220651 5430 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.221076 7684 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.221082 11108 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.221470 11108 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.221597 11111 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.222414 7548 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.222649 11108 raft_consensus.cc:1050] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER 4d4855c096ac4e1e936abf15834332ae to VOTER
I20240501 19:57:51.223161 11108 consensus_queue.cc:237] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:51.223425 10591 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.224126 6097 raft_consensus.cc:1261] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.224679 10566 consensus_queue.cc:1035] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.225028 6097 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.225431 10563 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.225535 5695 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.225688 5961 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.225867 10563 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.226173 5829 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.226578 7549 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.226604 11109 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.226999 10563 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.227361 10566 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.228682 11111 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.229053 11111 raft_consensus.cc:1013] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: attempt to promote peer 143c54da1456472dadbf1d5d65e0a6f6: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20240501 19:57:51.229229 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c6ba23a7b4d24e0aa077e47798e890b8 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:51.229457 6096 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.229916 5430 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.230767 5147 catalog_manager.cc:5484] T c6ba23a7b4d24e0aa077e47798e890b8 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 12 to 13, VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) evicted. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:51.232717 5695 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.234516 10566 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:51.234639 7682 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.236404 5961 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:51.237952 10520 raft_consensus.cc:1050] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 16737daa1d234f35826143a1aa29dc0b to VOTER
I20240501 19:57:51.238473 10520 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:51.238612 5152 catalog_manager.cc:5484] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 9 to 10, 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:51.239286 7684 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.240777 5562 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.241050 6096 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.242080 5430 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.242169 11168 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.242441 5296 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.242455 11168 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.242978 6097 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:51.243909 7684 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.243960 5829 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.244570 5430 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.244568 11017 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.244607 5296 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.244889 11110 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.245103 7549 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Committing config change with OpId 1.10: config changed from index 9 to 10, 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:51.245128 5829 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:51.245880 11107 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.246134 11107 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.247341 11110 raft_consensus.cc:1050] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171: attempting to promote NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 to VOTER
I20240501 19:57:51.247781 11110 consensus_queue.cc:237] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:51.247906 5430 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:51.249992 5962 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:51.250486 5430 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:51.250670 5694 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:51.251001 5827 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:51.251094 7684 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:51.251315 6095 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:51.251382 5295 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:51.251876 11107 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.251977 11207 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:51.252110 5296 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:51.252362 11107 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:51.252626 11107 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:51.252804 11017 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.253417 11168 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.253682 11168 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.253969 11107 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.254392 11107 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:51.254621 5694 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:51.256207 7682 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:51.256677 7549 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 LEARNER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:51.257004 11207 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:51.258436 7550 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.259514 11017 raft_consensus.cc:1013] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: attempt to promote peer 9782508a47ba4403a39ca49fff7677fa: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20240501 19:57:51.260015 5962 raft_consensus.cc:2932] T c6ba23a7b4d24e0aa077e47798e890b8 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.261193 7548 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.261314 7398 tablet_service.cc:1514] Processing DeleteTablet for tablet c6ba23a7b4d24e0aa077e47798e890b8 with delete_type TABLET_DATA_TOMBSTONED (TS 363f4ac4964f41a6ba0f98d1f487db83 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:52640
I20240501 19:57:51.261807 7416 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.262178 11168 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.262593 11215 tablet_replica.cc:330] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: stopping tablet replica
I20240501 19:57:51.262658 5295 raft_consensus.cc:1203] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.9->[1.10-1.11] Dedup: 1.10->[1.11-1.11]
I20240501 19:57:51.262900 11168 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.257356 11207 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:51.265409 10566 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.265969 7548 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.267619 11215 raft_consensus.cc:2229] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:51.267558 5426 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.267740 5829 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.268391 11215 raft_consensus.cc:2258] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:51.268362 5294 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.270074 5154 catalog_manager.cc:5484] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 10 to 11, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.270298 5962 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.270509 6097 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.271464 11168 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.274120 5562 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.275012 5695 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.275878 7684 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.275847 5829 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.274221 11027 raft_consensus.cc:1050] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0: attempting to promote NON_VOTER 363f4ac4964f41a6ba0f98d1f487db83 to VOTER
I20240501 19:57:51.278827 5149 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 12 to 13, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:51.279415 11027 consensus_queue.cc:237] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:51.279484 7418 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.281666 6096 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.281819 7682 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.282585 5430 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.287209 5562 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.287782 7682 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.288112 11226 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.288535 11226 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.288574 10361 raft_consensus.cc:1050] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3: attempting to promote NON_VOTER 9544b0e566c54a62a6135f3bff47c4f2 to VOTER
I20240501 19:57:51.288990 10361 consensus_queue.cc:237] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } }
I20240501 19:57:51.289631 11226 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.290706 7418 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 LEARNER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.291255 11226 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.291406 7549 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.291466 5695 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:51.291812 10641 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.293120 7549 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 19:57:51.293215 7684 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 19:57:51.293545 10916 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.293686 10539 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 19:57:51.294202 5961 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 LEARNER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.294315 6095 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 19:57:51.294606 5430 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 19:57:51.294977 6095 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.294997 11018 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.295166 10916 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.295492 7417 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 19:57:51.295784 5296 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.295933 5427 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.295940 10034 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.296348 10649 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.296311 6095 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.296844 10649 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.278685 5294 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.297286 5695 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.297524 5827 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 19:57:51.297936 10916 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 19:57:51.298156 11018 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 19:57:51.298177 7549 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.298822 5430 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEARNER]: Committing config change with OpId 1.13: config changed from index 12 to 13, 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.299109 10641 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.299453 10641 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.299695 10641 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.301662 5295 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 19:57:51.302996 5962 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.303364 10034 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.303529 11167 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 19:57:51.303925 11167 raft_consensus.cc:1013] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: attempt to promote peer 9782508a47ba4403a39ca49fff7677fa: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20240501 19:57:51.305699 7418 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.305541 10539 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.14: config changed from index 13 to 14, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.306038 10649 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.307138 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f0df1e5e85294f60b8de22053dd577f7 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:51.307823 5827 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.307823 6096 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.308568 5296 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.309612 5561 consensus_queue.cc:237] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:51.310609 5145 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 13 to 14, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:51.311395 5426 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEARNER]: Committing config change with OpId 1.14: config changed from index 13 to 14, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.313397 10916 raft_consensus.cc:1050] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 9782508a47ba4403a39ca49fff7677fa to VOTER
I20240501 19:57:51.313810 10916 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:51.314217 7682 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.314536 5694 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.314545 6097 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.314824 11207 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.315111 11206 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.315202 5543 tablet_service.cc:1514] Processing DeleteTablet for tablet f0df1e5e85294f60b8de22053dd577f7 with delete_type TABLET_DATA_TOMBSTONED (TS f5287d5ae4d84bd692b0f7985cd94171 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:43142
I20240501 19:57:51.315372 7549 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.315991 5295 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 19:57:51.316299 11017 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 19:57:51.316421 6093 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 19:57:51.316422 5430 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 19:57:51.316721 10520 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 19:57:51.316920 11109 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.316988 10520 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 19:57:51.317008 5827 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.317469 11109 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.318538 5827 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 19:57:51.318540 11109 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.318712 5295 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.319073 11206 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.319254 7548 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 19:57:51.319725 7417 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 19:57:51.320096 7682 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 19:57:51.320168 5426 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.320539 10520 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 19:57:51.320654 11017 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 19:57:51.320758 11018 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 19:57:51.320869 10539 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 19:57:51.321935 7548 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.321926 7417 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.322604 11207 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.323136 11207 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.323836 5296 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.324077 5829 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.325609 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9ecbb3dec08047478ded9809f3a0b859 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:51.326053 10034 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.325929 7682 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, VOTER f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: NON_VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.326889 5148 catalog_manager.cc:5484] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index 11 to 12, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.327167 6096 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.328399 5826 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.329238 10984 raft_consensus.cc:1050] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa: attempting to promote NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 to VOTER
I20240501 19:57:51.329301 5942 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ecbb3dec08047478ded9809f3a0b859 with delete_type TABLET_DATA_TOMBSTONED (TS 9544b0e566c54a62a6135f3bff47c4f2 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:42138
I20240501 19:57:51.329146 7684 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.329702 10984 consensus_queue.cc:237] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } }
I20240501 19:57:51.330053 11253 tablet_replica.cc:330] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2: stopping tablet replica
I20240501 19:57:51.330196 11253 raft_consensus.cc:2229] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:51.330721 11253 raft_consensus.cc:2258] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:51.331480 11253 ts_tablet_manager.cc:1875] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:51.331971 10520 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.332886 5825 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEARNER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.334394 5296 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.334645 7549 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.335381 5694 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.336079 5825 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.336557 5696 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.337956 5152 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 14 to 15, 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:51.338459 5561 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.338294 5430 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.338729 10984 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.339068 10984 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.339686 7684 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.339922 5561 consensus_queue.cc:237] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:51.340701 7417 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.340945 6096 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.341564 10984 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.342247 6093 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.342566 5296 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.342681 10984 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.342957 11108 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.343513 5295 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.343551 5696 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.343556 11167 raft_consensus.cc:1050] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae: attempting to promote NON_VOTER 1865147cd5a346ad8549f18fa8cc26c3 to VOTER
I20240501 19:57:51.344908 11167 consensus_queue.cc:237] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } }
I20240501 19:57:51.345206 11108 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.347106 6097 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.347609 10984 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.347751 5294 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.347880 5826 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.347965 6093 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.335847 5960 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:51.348131 11017 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.348274 11018 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.346750 5959 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.348397 11206 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.348657 11018 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.349411 5430 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.343153 5426 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.350342 11252 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.350765 11206 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.351933 11206 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.353477 7549 raft_consensus.cc:1261] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.353595 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9ecbb3dec08047478ded9809f3a0b859 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:51.354214 6096 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.353883 5426 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.354817 5826 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.355074 5151 catalog_manager.cc:5484] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 12 to 13, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:51.357417 7683 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.357535 7417 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.357685 11108 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.357836 7664 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ecbb3dec08047478ded9809f3a0b859 with delete_type TABLET_DATA_TOMBSTONED (TS 16737daa1d234f35826143a1aa29dc0b not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:44786
I20240501 19:57:51.358036 10566 consensus_queue.cc:1035] T 9ecbb3dec08047478ded9809f3a0b859 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.358642 11167 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.358669 5296 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.358947 11167 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.358971 5696 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.360738 5826 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEARNER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.361253 10539 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.361943 5430 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:51.362334 10539 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:51.362535 7549 raft_consensus.cc:2932] T 9ecbb3dec08047478ded9809f3a0b859 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:51.362612 7682 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.366451 11255 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:51.366900 11027 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.369163 7548 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, 9782508a47ba4403a39ca49fff7677fa (127.20.210.2) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false replace: true } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.370333 11253 ts_tablet_manager.cc:1888] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.11
I20240501 19:57:51.370414 11253 log.cc:1196] T 9ecbb3dec08047478ded9809f3a0b859 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/wals/9ecbb3dec08047478ded9809f3a0b859
I20240501 19:57:51.370802 5562 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.370865 6096 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.370968 5152 catalog_manager.cc:5484] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:51.372049 5141 catalog_manager.cc:4830] TS 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085): tablet 9ecbb3dec08047478ded9809f3a0b859 (table rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]) successfully deleted
I20240501 19:57:51.373113 7417 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.374312 5429 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.375523 7682 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.377661 11167 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.379765 5430 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.379757 5959 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.381425 5696 consensus_queue.cc:237] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 1.15, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } }
I20240501 19:57:51.383059 5294 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 19:57:51.382993 5296 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.383911 7418 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.384112 5430 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 19:57:51.383913 5145 catalog_manager.cc:5484] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 11 to 12, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:51.384696 6096 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 19:57:51.385057 10520 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 19:57:51.385645 11016 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 19:57:51.385890 11016 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 19:57:51.387953 7418 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 19:57:51.388489 7682 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 19:57:51.388566 5826 raft_consensus.cc:1261] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240501 19:57:51.388681 10520 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.390115 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f0df1e5e85294f60b8de22053dd577f7 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:51.390651 5561 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.390805 6096 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.391130 11249 tablet_replica.cc:330] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171: stopping tablet replica
I20240501 19:57:51.391242 11249 raft_consensus.cc:2229] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:51.391603 11249 raft_consensus.cc:2258] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:51.391485 5430 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.392648 10641 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.393460 11252 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.393560 5296 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.394717 10916 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 19:57:51.395210 11167 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 19:57:51.395707 6095 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.396289 11016 consensus_queue.cc:1035] T f0df1e5e85294f60b8de22053dd577f7 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240501 19:57:51.396904 5825 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.397516 5959 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.397881 5826 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.398017 7683 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.400187 7418 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.400249 5825 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.400355 5154 catalog_manager.cc:5484] T f0df1e5e85294f60b8de22053dd577f7 P a881063bd86945a8871c7c102d8235c8 reported cstate change: config changed from index 15 to 16, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.401696 5695 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.401885 5152 catalog_manager.cc:5484] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 8 to 9, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9782508a47ba4403a39ca49fff7677fa" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.402096 7548 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.404685 5296 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.405799 7418 raft_consensus.cc:2932] T f0df1e5e85294f60b8de22053dd577f7 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } }
I20240501 19:57:51.406029 5149 catalog_manager.cc:5484] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 reported cstate change: config changed from index 11 to 12, 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.407258 5294 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.408537 5695 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.410884 7682 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.413414 5430 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.415050 7530 tablet_service.cc:1514] Processing DeleteTablet for tablet f0df1e5e85294f60b8de22053dd577f7 with delete_type TABLET_DATA_TOMBSTONED (TS 143c54da1456472dadbf1d5d65e0a6f6 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:36080
I20240501 19:57:51.415277 11050 tablet_replica.cc:330] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: stopping tablet replica
I20240501 19:57:51.415405 11050 raft_consensus.cc:2229] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:51.415611 11050 raft_consensus.cc:2258] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:51.415913 11249 ts_tablet_manager.cc:1875] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:51.419111 11050 ts_tablet_manager.cc:1875] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:51.419350 11249 ts_tablet_manager.cc:1888] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.13
I20240501 19:57:51.419423 11249 log.cc:1196] T f0df1e5e85294f60b8de22053dd577f7 P f5287d5ae4d84bd692b0f7985cd94171: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-2/wal/wals/f0df1e5e85294f60b8de22053dd577f7
I20240501 19:57:51.420935 11050 ts_tablet_manager.cc:1888] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.15
I20240501 19:57:51.420993 11050 log.cc:1196] T f0df1e5e85294f60b8de22053dd577f7 P 143c54da1456472dadbf1d5d65e0a6f6: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-8/wal/wals/f0df1e5e85294f60b8de22053dd577f7
I20240501 19:57:51.421586 5141 catalog_manager.cc:4830] TS f5287d5ae4d84bd692b0f7985cd94171 (127.20.210.3:44275): tablet f0df1e5e85294f60b8de22053dd577f7 (table rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]) successfully deleted
I20240501 19:57:51.422454 5139 catalog_manager.cc:4830] TS 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): tablet f0df1e5e85294f60b8de22053dd577f7 (table rebalance_test_table_5 [id=2c09ba5954bd4ccba7ff8160b6631756]) successfully deleted
I20240501 19:57:51.423035 11215 ts_tablet_manager.cc:1875] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:51.425132 11215 ts_tablet_manager.cc:1888] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 19:57:51.425503 11215 log.cc:1196] T c6ba23a7b4d24e0aa077e47798e890b8 P 363f4ac4964f41a6ba0f98d1f487db83: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-7/wal/wals/c6ba23a7b4d24e0aa077e47798e890b8
I20240501 19:57:51.426077 5139 catalog_manager.cc:4830] TS 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8:41983): tablet c6ba23a7b4d24e0aa077e47798e890b8 (table rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]) successfully deleted
I20240501 19:57:51.432296 6097 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.432657 5828 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.435042 7684 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.435717 5695 consensus_queue.cc:237] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } }
I20240501 19:57:51.437584 7418 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.437837 7684 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.438048 5430 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.438060 5296 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.438390 11016 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.438516 11017 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.438611 5828 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.439231 11017 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.439352 11016 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.439661 11016 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.441609 6097 raft_consensus.cc:1261] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 4d4855c096ac4e1e936abf15834332ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.441999 11016 consensus_queue.cc:1035] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.442565 11196 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.443013 7684 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.444485 5141 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet cb5540ce8d3343a68a8bf7d8fff00589 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:51.445295 5296 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.446009 5145 catalog_manager.cc:5484] T cb5540ce8d3343a68a8bf7d8fff00589 P 4d4855c096ac4e1e936abf15834332ae reported cstate change: config changed from index 12 to 13, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New cstate: current_term: 1 leader_uuid: "4d4855c096ac4e1e936abf15834332ae" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:51.447062 7418 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.447207 5430 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.450016 5828 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.450990 11271 tablet_replica.cc:330] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: stopping tablet replica
I20240501 19:57:51.451345 11271 raft_consensus.cc:2229] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:51.451648 11271 raft_consensus.cc:2258] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:51.453563 5962 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.454458 5942 tablet_service.cc:1514] Processing DeleteTablet for tablet cb5540ce8d3343a68a8bf7d8fff00589 with delete_type TABLET_DATA_TOMBSTONED (TS 9544b0e566c54a62a6135f3bff47c4f2 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:42138
I20240501 19:57:51.454638 11253 tablet_replica.cc:330] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2: stopping tablet replica
I20240501 19:57:51.454758 11253 raft_consensus.cc:2229] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:51.455052 11253 raft_consensus.cc:2258] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:51.456238 11253 ts_tablet_manager.cc:1875] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:51.458401 6093 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1865147cd5a346ad8549f18fa8cc26c3 (127.20.210.5) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } attrs { replace: true } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.460530 5962 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 363f4ac4964f41a6ba0f98d1f487db83 (127.20.210.8) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.461501 11271 ts_tablet_manager.cc:1875] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:51.463928 11271 ts_tablet_manager.cc:1888] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 19:57:51.464012 11271 log.cc:1196] T 9ecbb3dec08047478ded9809f3a0b859 P 16737daa1d234f35826143a1aa29dc0b: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal/wals/9ecbb3dec08047478ded9809f3a0b859
I20240501 19:57:51.464601 11253 ts_tablet_manager.cc:1888] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 19:57:51.464677 11253 log.cc:1196] T cb5540ce8d3343a68a8bf7d8fff00589 P 9544b0e566c54a62a6135f3bff47c4f2: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-5/wal/wals/cb5540ce8d3343a68a8bf7d8fff00589
I20240501 19:57:51.465160 5141 catalog_manager.cc:4830] TS 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6:34085): tablet cb5540ce8d3343a68a8bf7d8fff00589 (table rebalance_test_table_6 [id=2ef06ac769174b8094fde568444beeda]) successfully deleted
I20240501 19:57:51.466050 5296 consensus_queue.cc:237] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } }
I20240501 19:57:51.467538 5430 consensus_queue.cc:237] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } }
I20240501 19:57:51.467898 5561 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.468178 5429 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.468276 11226 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.468580 11226 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.469600 5561 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.469600 5826 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.469736 5961 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.469960 11227 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.470096 11287 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.470167 6093 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.470798 11287 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.470968 11227 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.471364 6097 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.471696 11227 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.472199 5294 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.473222 5962 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.473501 7684 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.473537 11255 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.473826 11252 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.474226 11255 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.474980 6097 raft_consensus.cc:2932] T cb5540ce8d3343a68a8bf7d8fff00589 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 9544b0e566c54a62a6135f3bff47c4f2 (127.20.210.6) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "4d4855c096ac4e1e936abf15834332ae" member_type: VOTER last_known_addr { host: "127.20.210.4" port: 46519 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.475586 11228 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.475929 5561 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.478296 5138 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 913563ed5f904cc28ff15148d9a5e5a2 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:51.478613 5826 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.479988 5825 consensus_queue.cc:237] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } }
I20240501 19:57:51.480451 5430 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.481205 5828 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.481551 7418 raft_consensus.cc:1261] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer ad52038f1d38489ea464a8fbc72daac0: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240501 19:57:51.481738 7417 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.481935 11228 consensus_queue.cc:1035] T 913563ed5f904cc28ff15148d9a5e5a2 P ad52038f1d38489ea464a8fbc72daac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.481868 6097 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.482405 11232 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.482940 5961 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.482993 7417 raft_consensus.cc:2932] T 913563ed5f904cc28ff15148d9a5e5a2 P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.483777 11252 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.484198 5296 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.484959 5152 catalog_manager.cc:5484] T 913563ed5f904cc28ff15148d9a5e5a2 P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 9 to 10, VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) evicted. New cstate: current_term: 1 leader_uuid: "ad52038f1d38489ea464a8fbc72daac0" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } }
I20240501 19:57:51.486558 5961 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.486851 5138 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0681575c2bb142e0915be3dc75858f87 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:51.488025 5561 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.488340 11287 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240501 19:57:51.489398 6093 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.489436 5828 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.490960 7664 tablet_service.cc:1514] Processing DeleteTablet for tablet 913563ed5f904cc28ff15148d9a5e5a2 with delete_type TABLET_DATA_TOMBSTONED (TS 16737daa1d234f35826143a1aa29dc0b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:44786
I20240501 19:57:51.491019 5961 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.492240 5145 catalog_manager.cc:5484] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 reported cstate change: config changed from index 9 to 10, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "9782508a47ba4403a39ca49fff7677fa" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.492409 7548 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.492646 11232 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.492789 5296 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.493010 11232 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.493561 11232 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.494328 11232 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.494540 5674 tablet_service.cc:1514] Processing DeleteTablet for tablet 0681575c2bb142e0915be3dc75858f87 with delete_type TABLET_DATA_TOMBSTONED (TS 4d4855c096ac4e1e936abf15834332ae not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:36532
I20240501 19:57:51.495115 11321 tablet_replica.cc:330] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae: stopping tablet replica
I20240501 19:57:51.495396 6093 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.496623 7684 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:51.498850 11321 raft_consensus.cc:2229] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:51.499132 11321 raft_consensus.cc:2258] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:51.500010 11321 ts_tablet_manager.cc:1875] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:51.501646 10641 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.502029 5430 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240501 19:57:51.502622 11321 ts_tablet_manager.cc:1888] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:51.502749 11321 log.cc:1196] T 0681575c2bb142e0915be3dc75858f87 P 4d4855c096ac4e1e936abf15834332ae: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/0681575c2bb142e0915be3dc75858f87
I20240501 19:57:51.504841 7417 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.505072 5296 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.504518 5141 catalog_manager.cc:4830] TS 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519): tablet 0681575c2bb142e0915be3dc75858f87 (table rebalance_test_table_1 [id=8a74b9a0301c4fdba13c36f10df45d81]) successfully deleted
I20240501 19:57:51.504557 5139 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d8a84349ba37496289351ac689f77eab with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240501 19:57:51.504298 5147 catalog_manager.cc:5484] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 12 to 13, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:51.507370 10361 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240501 19:57:51.507676 5961 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.509343 5430 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.509863 5674 tablet_service.cc:1514] Processing DeleteTablet for tablet d8a84349ba37496289351ac689f77eab with delete_type TABLET_DATA_TOMBSTONED (TS 4d4855c096ac4e1e936abf15834332ae not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:36532
I20240501 19:57:51.509972 6093 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.510586 11321 tablet_replica.cc:330] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae: stopping tablet replica
I20240501 19:57:51.510732 11321 raft_consensus.cc:2229] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:51.513706 7548 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 4d4855c096ac4e1e936abf15834332ae (127.20.210.4) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:51.512326 11321 raft_consensus.cc:2258] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:51.515717 11321 ts_tablet_manager.cc:1875] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:51.516448 5139 catalog_manager.cc:4830] TS 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): tablet 9ecbb3dec08047478ded9809f3a0b859 (table rebalance_test_table_4 [id=47cd34f1f721421caa2c57f78717980a]) successfully deleted
I20240501 19:57:51.516551 11271 tablet_replica.cc:330] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: stopping tablet replica
I20240501 19:57:51.516706 11271 raft_consensus.cc:2229] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240501 19:57:51.517019 11271 raft_consensus.cc:2258] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240501 19:57:51.517767 11271 ts_tablet_manager.cc:1875] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240501 19:57:51.518110 11321 ts_tablet_manager.cc:1888] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240501 19:57:51.518214 11321 log.cc:1196] T d8a84349ba37496289351ac689f77eab P 4d4855c096ac4e1e936abf15834332ae: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-3/wal/wals/d8a84349ba37496289351ac689f77eab
I20240501 19:57:51.518774 5141 catalog_manager.cc:4830] TS 4d4855c096ac4e1e936abf15834332ae (127.20.210.4:46519): tablet d8a84349ba37496289351ac689f77eab (table rebalance_test_table_3 [id=93dd9c8724444b398e4a4356fe47019e]) successfully deleted
I20240501 19:57:51.519824 11271 ts_tablet_manager.cc:1888] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240501 19:57:51.519891 11271 log.cc:1196] T 913563ed5f904cc28ff15148d9a5e5a2 P 16737daa1d234f35826143a1aa29dc0b: Deleting WAL directory at /tmp/dist-test-taskNtLy6c/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714593358208957-21320-0/raft_consensus-itest-cluster/ts-9/wal/wals/913563ed5f904cc28ff15148d9a5e5a2
I20240501 19:57:51.520309 5139 catalog_manager.cc:4830] TS 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): tablet 913563ed5f904cc28ff15148d9a5e5a2 (table rebalance_test_table_2 [id=dda509fc301f4fd789a0080827e9f7f2]) successfully deleted
I20240501 19:57:51.587291 5408 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:51.590348 7664 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:51.595055 7398 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:51.595355 5276 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:51.598546 5543 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:51.598837 6076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:51.603272 5809 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:51.613603 5674 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:51.632090 7530 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:51.651283 5942 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:51.741135 7530 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:51.742075 5674 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:51.743984 5809 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:51.754827 7398 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:51.758934 5942 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:51.763491 7664 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:51.772512 5543 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:51.786871 6076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:51.789300 5408 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:51.804826 5276 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:51.987387 5152 ts_manager.cc:292] Set tserver state for a881063bd86945a8871c7c102d8235c8 to MAINTENANCE_MODE
I20240501 19:57:52.190016 7398 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:52.190344 7664 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:52.192159 5674 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:52.195742 5809 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:52.199136 5942 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:52.214403 5276 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:52.216439 7530 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:52.222366 5543 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:52.243973 6076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:52.244601 5408 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:52.362011 7530 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:52.366636 5408 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:52.368891 5276 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:52.370117 6076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:52.371446 5674 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:52.371816 5942 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:52.372771 5543 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:52.375451 5809 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:52.376379 7664 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:52.409775 7398 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:52.538790 7530 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:52.543385 5942 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:52.550065 7398 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:52.560475 5276 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:52.563019 5809 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240501 19:57:52.569083 7664 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:52.575237 6076 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240501 19:57:52.576598 5674 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:52.586887 5543 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240501 19:57:52.598155 5408 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240501 19:57:52.633739 5430 consensus_queue.cc:237] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } }
I20240501 19:57:52.635424 6097 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:52.635478 5962 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:52.635600 5294 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:52.635600 5828 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:52.635954 5561 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:52.636031 11569 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:52.636179 11549 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:52.636307 7684 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240501 19:57:52.636371 11570 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:52.637532 11255 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:52.637740 11569 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:52.637859 11549 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240501 19:57:52.646709 11255 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:52.647293 5828 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:52.647816 5561 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:52.649802 7684 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:52.650432 5152 catalog_manager.cc:5484] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 10 to 11. New cstate: current_term: 1 leader_uuid: "9782508a47ba4403a39ca49fff7677fa" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:52.650439 5294 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:52.653424 5962 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:52.655553 6097 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } }
I20240501 19:57:52.658681 5430 consensus_queue.cc:237] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } }
I20240501 19:57:52.661262 5561 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:52.661276 5962 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:52.661540 6097 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:52.661635 5828 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:52.661944 7684 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:52.662024 11570 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:52.662348 11255 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:52.662693 11569 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:52.662734 5294 raft_consensus.cc:1261] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 9782508a47ba4403a39ca49fff7677fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240501 19:57:52.663041 11255 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:52.663329 11255 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:52.664863 11570 consensus_queue.cc:1035] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240501 19:57:52.665174 5562 consensus_queue.cc:237] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } }
I20240501 19:57:52.666083 11569 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:52.667677 5138 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 0681575c2bb142e0915be3dc75858f87 with cas_config_opid_index 11: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:52.668167 5294 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:52.668493 5828 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:52.668551 7548 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:52.668573 11421 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:52.668906 11421 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:52.669168 11421 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:52.669081 5148 catalog_manager.cc:5484] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa reported cstate change: config changed from index 11 to 12, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New cstate: current_term: 1 leader_uuid: "9782508a47ba4403a39ca49fff7677fa" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240501 19:57:52.669559 6093 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:52.669898 5961 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:52.670274 11592 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:52.670686 6097 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240501 19:57:52.671789 5364 consensus_peers.cc:485] T 0681575c2bb142e0915be3dc75858f87 P 9782508a47ba4403a39ca49fff7677fa -> Peer 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9:45861): Couldn't send request to peer 143c54da1456472dadbf1d5d65e0a6f6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0681575c2bb142e0915be3dc75858f87. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:52.672243 5828 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:52.673135 5430 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240501 19:57:52.673338 11593 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:52.673168 5294 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:52.674582 5962 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:52.676793 5561 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:52.678100 11593 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:52.678459 7684 raft_consensus.cc:2932] T 0681575c2bb142e0915be3dc75858f87 P 16737daa1d234f35826143a1aa29dc0b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER 143c54da1456472dadbf1d5d65e0a6f6 (127.20.210.9) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: false } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: NON_VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: true } } }
I20240501 19:57:52.678630 6093 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:52.680204 11206 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240501 19:57:52.680290 5962 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:52.680982 5828 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:52.681315 5430 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:52.683264 7548 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:52.683524 5148 catalog_manager.cc:5484] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 reported cstate change: config changed from index 7 to 8. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:52.685986 5296 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } }
I20240501 19:57:52.687794 5561 consensus_queue.cc:237] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:52.689927 6093 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:52.689957 5962 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:52.690405 7548 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
W20240501 19:57:52.690634 5497 consensus_peers.cc:485] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 80fdea49202c4ab5b3972b2c9ad7cff5. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:52.690841 11593 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:52.690965 5828 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:52.690968 11421 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:52.691248 11421 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:52.691334 5430 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:52.692193 11594 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:52.692502 11594 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:52.694041 5296 raft_consensus.cc:1261] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer f5287d5ae4d84bd692b0f7985cd94171: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240501 19:57:52.694080 11421 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.694489 11206 consensus_queue.cc:1035] T 80fdea49202c4ab5b3972b2c9ad7cff5 P f5287d5ae4d84bd692b0f7985cd94171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240501 19:57:52.695415 5296 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.696102 5141 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 80fdea49202c4ab5b3972b2c9ad7cff5 with cas_config_opid_index 8: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:52.696709 6093 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.696766 5430 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.697634 5147 catalog_manager.cc:5484] T 80fdea49202c4ab5b3972b2c9ad7cff5 P ad52038f1d38489ea464a8fbc72daac0 reported cstate change: config changed from index 8 to 9, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "f5287d5ae4d84bd692b0f7985cd94171" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.698030 5962 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.699419 5828 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.700449 7548 raft_consensus.cc:2932] T 80fdea49202c4ab5b3972b2c9ad7cff5 P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "f5287d5ae4d84bd692b0f7985cd94171" member_type: VOTER last_known_addr { host: "127.20.210.3" port: 44275 } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.702368 5828 consensus_queue.cc:237] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } }
I20240501 19:57:52.705955 7548 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 19:57:52.706629 6093 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 19:57:52.706907 5430 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 19:57:52.706914 5962 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 19:57:52.707118 5296 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 19:57:52.707401 11236 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 19:57:52.707527 7417 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240501 19:57:52.707566 11609 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 19:57:52.707782 11611 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 19:57:52.708003 11236 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 19:57:52.708132 11431 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 19:57:52.708287 11610 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240501 19:57:52.710902 11236 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:52.711424 7417 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:52.711766 5430 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:52.711830 5962 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:52.713888 7548 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:52.714342 5147 catalog_manager.cc:5484] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 reported cstate change: config changed from index 13 to 14. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240501 19:57:52.714428 6093 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:52.717058 5296 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } }
I20240501 19:57:52.717504 5828 consensus_queue.cc:237] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } }
I20240501 19:57:52.719583 7417 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 19:57:52.719595 5430 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 19:57:52.719959 11610 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20240501 19:57:52.720225 5762 consensus_peers.cc:485] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 -> Peer 16737daa1d234f35826143a1aa29dc0b (127.20.210.10:40603): Couldn't send request to peer 16737daa1d234f35826143a1aa29dc0b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8a84349ba37496289351ac689f77eab. This is attempt 1: this message will repeat every 5th retry.
I20240501 19:57:52.720214 11611 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 19:57:52.720486 7548 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 19:57:52.720669 6093 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 19:57:52.721009 11236 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 19:57:52.721048 5962 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 19:57:52.721657 11611 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 19:57:52.721704 5296 raft_consensus.cc:1261] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Refusing update from remote peer 1865147cd5a346ad8549f18fa8cc26c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240501 19:57:52.722273 11611 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 19:57:52.723351 11610 consensus_queue.cc:1035] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240501 19:57:52.725106 11236 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 1865147cd5a346ad8549f18fa8cc26c3 [term 1 LEADER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.725518 7548 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.726408 5139 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet d8a84349ba37496289351ac689f77eab with cas_config_opid_index 14: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240501 19:57:52.727547 7417 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 363f4ac4964f41a6ba0f98d1f487db83 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.727547 5962 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9544b0e566c54a62a6135f3bff47c4f2 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.727841 5148 catalog_manager.cc:5484] T d8a84349ba37496289351ac689f77eab P 143c54da1456472dadbf1d5d65e0a6f6 reported cstate change: config changed from index 14 to 15, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New cstate: current_term: 1 leader_uuid: "1865147cd5a346ad8549f18fa8cc26c3" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.729455 6093 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P a881063bd86945a8871c7c102d8235c8 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.731853 5430 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P 9782508a47ba4403a39ca49fff7677fa [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:52.731916 5296 raft_consensus.cc:2932] T d8a84349ba37496289351ac689f77eab P ad52038f1d38489ea464a8fbc72daac0 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, NON_VOTER 16737daa1d234f35826143a1aa29dc0b (127.20.210.10) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "ad52038f1d38489ea464a8fbc72daac0" member_type: VOTER last_known_addr { host: "127.20.210.1" port: 46285 } } peers { permanent_uuid: "9782508a47ba4403a39ca49fff7677fa" member_type: VOTER last_known_addr { host: "127.20.210.2" port: 35469 } } peers { permanent_uuid: "1865147cd5a346ad8549f18fa8cc26c3" member_type: VOTER last_known_addr { host: "127.20.210.5" port: 39499 } } peers { permanent_uuid: "a881063bd86945a8871c7c102d8235c8" member_type: VOTER last_known_addr { host: "127.20.210.7" port: 39815 } attrs { replace: true } } peers { permanent_uuid: "363f4ac4964f41a6ba0f98d1f487db83" member_type: VOTER last_known_addr { host: "127.20.210.8" port: 41983 } attrs { promote: false } } peers { permanent_uuid: "143c54da1456472dadbf1d5d65e0a6f6" member_type: VOTER last_known_addr { host: "127.20.210.9" port: 45861 } attrs { promote: false } } peers { permanent_uuid: "9544b0e566c54a62a6135f3bff47c4f2" member_type: VOTER last_known_addr { host: "127.20.210.6" port: 34085 } attrs { promote: false } } peers { permanent_uuid: "16737daa1d234f35826143a1aa29dc0b" member_type: NON_VOTER last_known_addr { host: "127.20.210.10" port: 40603 } attrs { promote: true } } }
I20240501 19:57:53.040298 11652 ts_tablet_manager.cc:902] T 0681575c2bb142e0915be3dc75858f87 P 143c54da1456472dadbf1d5d65e0a6f6: Initiating tablet copy from peer 9782508a47ba4403a39ca49fff7677fa (127.20.210.2:35469)
F20240501 19:57:53.040491 11652 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-064f698b376e407da78c7d439553896a.queue_time_us, tablet-download-pool-064f698b376e407da78c7d439553896a.queue_time_us) (tablet-download-pool-064f698b376e407da78c7d439553896a.run_cpu_time_us, tablet-download-pool-064f698b376e407da78c7d439553896a.run_cpu_time_us) (tablet-download-pool-064f698b376e407da78c7d439553896a.run_wall_time_us, tablet-download-pool-064f698b376e407da78c7d439553896a.run_wall_time_us) (tablet-download-pool-0681575c2bb142e0915be3dc75858f87.queue_time_us, tablet-download-pool-0681575c2bb142e0915be3dc75858f87.queue_time_us) (tablet-download-pool-1211024f45e142979fc1e5e8fff010ce.queue_time_us, tablet-download-pool-1211024f45e142979fc1e5e8fff010ce.queue_time_us) (tablet-download-pool-1211024f45e142979fc1e5e8fff010ce.run_cpu_time_us, tablet-download-pool-1211024f45e142979fc1e5e8fff010ce.run_cpu_time_us) (tablet-download-pool-1211024f45e142979fc1e5e8fff010ce.run_wall_time_us, tablet-download-pool-1211024f45e142979fc1e5e8fff010ce.run_wall_time_us) (tablet-download-pool-4a713bc0643d4bdf91f669dde94f63f8.queue_time_us, tablet-download-pool-4a713bc0643d4bdf91f669dde94f63f8.queue_time_us) (tablet-download-pool-4a713bc0643d4bdf91f669dde94f63f8.run_cpu_time_us, tablet-download-pool-4a713bc0643d4bdf91f669dde94f63f8.run_cpu_time_us) (tablet-download-pool-4a713bc0643d4bdf91f669dde94f63f8.run_wall_time_us, tablet-download-pool-4a713bc0643d4bdf91f669dde94f63f8.run_wall_time_us) (tablet-download-pool-80fdea49202c4ab5b3972b2c9ad7cff5.queue_time_us, tablet-download-pool-80fdea49202c4ab5b3972b2c9ad7cff5.queue_time_us) (tablet-download-pool-80fdea49202c4ab5b3972b2c9ad7cff5.run_cpu_time_us, tablet-download-pool-80fdea49202c4ab5b3972b2c9ad7cff5.run_cpu_time_us) (tablet-download-pool-80fdea49202c4ab5b3972b2c9ad7cff5.run_wall_time_us, tablet-download-pool-80fdea49202c4ab5b3972b2c9ad7cff5.run_wall_time_us) (tablet-download-pool-85d29eb6552945a7879ee1a7e12919f0.queue_time_us, tablet-download-pool-85d29eb6552945a7879ee1a7e12919f0.queue_time_us) (tablet-download-pool-85d29eb6552945a7879ee1a7e12919f0.run_cpu_time_us, tablet-download-pool-85d29eb6552945a7879ee1a7e12919f0.run_cpu_time_us) (tablet-download-pool-85d29eb6552945a7879ee1a7e12919f0.run_wall_time_us, tablet-download-pool-85d29eb6552945a7879ee1a7e12919f0.run_wall_time_us) (tablet-download-pool-95e8db3b5d81468098c01c32c4d22e80.queue_time_us, tablet-download-pool-95e8db3b5d81468098c01c32c4d22e80.queue_time_us) (tablet-download-pool-95e8db3b5d81468098c01c32c4d22e80.run_cpu_time_us, tablet-download-pool-95e8db3b5d81468098c01c32c4d22e80.run_cpu_time_us) (tablet-download-pool-95e8db3b5d81468098c01c32c4d22e80.run_wall_time_us, tablet-download-pool-95e8db3b5d8146809